Jul 23 13:05:31.765239 ip-10-0-130-1 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 23 13:05:31.765255 ip-10-0-130-1 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 23 13:05:31.765264 ip-10-0-130-1 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 23 13:05:31.765582 ip-10-0-130-1 systemd[1]: Failed to start Kubernetes Kubelet. Jul 23 13:05:41.798999 ip-10-0-130-1 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 23 13:05:41.799015 ip-10-0-130-1 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 4d830b6d54b94db390ab6499406cd123 -- Jul 23 13:07:42.631652 ip-10-0-130-1 systemd[1]: Starting Kubernetes Kubelet... Jul 23 13:07:43.052595 ip-10-0-130-1 kubenswrapper[2531]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 13:07:43.052595 ip-10-0-130-1 kubenswrapper[2531]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 23 13:07:43.052595 ip-10-0-130-1 kubenswrapper[2531]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 13:07:43.052595 ip-10-0-130-1 kubenswrapper[2531]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 23 13:07:43.052595 ip-10-0-130-1 kubenswrapper[2531]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 13:07:43.053493 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.053403 2531 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 23 13:07:43.060845 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060827 2531 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 13:07:43.060930 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060847 2531 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 13:07:43.060930 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060853 2531 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 13:07:43.060930 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060856 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 13:07:43.060930 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060859 2531 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 13:07:43.060930 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060862 2531 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 13:07:43.060930 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060865 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 13:07:43.060930 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060868 2531 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 13:07:43.060930 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060871 2531 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 13:07:43.060930 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060874 2531 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 13:07:43.060930 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060877 2531 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 13:07:43.060930 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060880 2531 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 13:07:43.060930 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060883 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 13:07:43.060930 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060885 2531 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 13:07:43.060930 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060888 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 13:07:43.060930 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060890 2531 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 13:07:43.060930 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060893 2531 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 13:07:43.060930 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060900 2531 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 13:07:43.060930 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060903 2531 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 13:07:43.060930 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060905 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 13:07:43.061418 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060908 2531 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 13:07:43.061418 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060910 2531 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 13:07:43.061418 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060912 2531 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 13:07:43.061418 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060915 2531 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 13:07:43.061418 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060917 2531 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 13:07:43.061418 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060920 2531 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 13:07:43.061418 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060922 2531 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 13:07:43.061418 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060924 2531 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 13:07:43.061418 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060927 2531 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 13:07:43.061418 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060930 2531 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 13:07:43.061418 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060933 2531 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 13:07:43.061418 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060935 2531 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 13:07:43.061418 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060938 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 13:07:43.061418 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060941 2531 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 13:07:43.061418 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060944 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 13:07:43.061418 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060946 2531 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 13:07:43.061418 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060950 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 13:07:43.061418 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060952 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 13:07:43.061418 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060954 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 13:07:43.061418 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060957 2531 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 13:07:43.061888 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060960 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 13:07:43.061888 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060964 2531 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 13:07:43.061888 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060967 2531 feature_gate.go:328] unrecognized feature gate: Example Jul 23 13:07:43.061888 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060969 2531 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 13:07:43.061888 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060973 2531 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 13:07:43.061888 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060975 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 13:07:43.061888 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060978 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 13:07:43.061888 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060980 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 13:07:43.061888 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060982 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 13:07:43.061888 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060985 2531 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 13:07:43.061888 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060987 2531 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 13:07:43.061888 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060990 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 13:07:43.061888 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060992 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 13:07:43.061888 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060996 2531 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 13:07:43.061888 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.060999 2531 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 13:07:43.061888 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061001 2531 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 13:07:43.061888 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061004 2531 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 13:07:43.061888 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061006 2531 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 13:07:43.061888 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061009 2531 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 13:07:43.062337 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061011 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 13:07:43.062337 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061014 2531 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 13:07:43.062337 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061016 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 13:07:43.062337 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061018 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 13:07:43.062337 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061021 2531 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 13:07:43.062337 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061025 2531 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 13:07:43.062337 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061027 2531 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 13:07:43.062337 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061031 2531 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 13:07:43.062337 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061035 2531 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 13:07:43.062337 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061037 2531 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 13:07:43.062337 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061040 2531 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 13:07:43.062337 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061042 2531 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 13:07:43.062337 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061045 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 13:07:43.062337 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061047 2531 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 13:07:43.062337 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061050 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 13:07:43.062337 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061052 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 13:07:43.062337 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061055 2531 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 13:07:43.062337 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061058 2531 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 13:07:43.062337 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061060 2531 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 13:07:43.062337 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061063 2531 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 13:07:43.062835 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061065 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 13:07:43.062835 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061067 2531 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 13:07:43.062835 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061070 2531 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 13:07:43.062835 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061072 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 13:07:43.062835 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061075 2531 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 13:07:43.062835 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061077 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 13:07:43.062835 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061080 2531 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 13:07:43.062835 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061083 2531 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 13:07:43.062835 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061085 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 13:07:43.062835 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061087 2531 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 13:07:43.062835 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061090 2531 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 13:07:43.062835 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061092 2531 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 13:07:43.062835 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061095 2531 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 13:07:43.062835 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061098 2531 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 13:07:43.062835 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061100 2531 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 13:07:43.062835 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061102 2531 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 13:07:43.062835 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061105 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 13:07:43.062835 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061107 2531 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 13:07:43.062835 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061109 2531 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 13:07:43.062835 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061112 2531 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 13:07:43.063322 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061830 2531 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 13:07:43.063322 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061837 2531 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 13:07:43.063322 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061840 2531 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 13:07:43.063322 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061842 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 13:07:43.063322 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061845 2531 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 13:07:43.063322 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061847 2531 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 13:07:43.063322 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061850 2531 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 13:07:43.063322 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061853 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 13:07:43.063322 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061856 2531 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 13:07:43.063322 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061858 2531 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 13:07:43.063322 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061861 2531 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 13:07:43.063322 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061864 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 13:07:43.063322 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061866 2531 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 13:07:43.063322 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061869 2531 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 13:07:43.063322 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061873 2531 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 13:07:43.063322 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061877 2531 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 13:07:43.063322 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061880 2531 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 13:07:43.063322 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061882 2531 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 13:07:43.063322 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061886 2531 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 13:07:43.063829 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061889 2531 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 13:07:43.063829 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061892 2531 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 13:07:43.063829 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061894 2531 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 13:07:43.063829 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061897 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 13:07:43.063829 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061900 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 13:07:43.063829 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061903 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 13:07:43.063829 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061905 2531 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 13:07:43.063829 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061908 2531 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 13:07:43.063829 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061910 2531 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 13:07:43.063829 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061914 2531 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 13:07:43.063829 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061916 2531 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 13:07:43.063829 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061918 2531 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 13:07:43.063829 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061921 2531 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 13:07:43.063829 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061923 2531 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 13:07:43.063829 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061926 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 13:07:43.063829 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061928 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 13:07:43.063829 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061931 2531 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 13:07:43.063829 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061935 2531 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 13:07:43.063829 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061939 2531 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 13:07:43.064282 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061942 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 13:07:43.064282 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061945 2531 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 13:07:43.064282 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061948 2531 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 13:07:43.064282 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061950 2531 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 13:07:43.064282 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061953 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 13:07:43.064282 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061956 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 13:07:43.064282 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061960 2531 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 13:07:43.064282 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061963 2531 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 13:07:43.064282 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061966 2531 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 13:07:43.064282 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061969 2531 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 13:07:43.064282 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061971 2531 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 13:07:43.064282 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061974 2531 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 13:07:43.064282 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061977 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 13:07:43.064282 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061979 2531 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 13:07:43.064282 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061982 2531 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 13:07:43.064282 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061985 2531 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 13:07:43.064282 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061988 2531 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 13:07:43.064282 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061990 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 13:07:43.064282 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061993 2531 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 13:07:43.064282 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061996 2531 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 13:07:43.064788 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.061998 2531 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 13:07:43.064788 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062000 2531 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 13:07:43.064788 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062003 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 13:07:43.064788 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062006 2531 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 13:07:43.064788 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062008 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 13:07:43.064788 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062010 2531 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 13:07:43.064788 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062014 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 13:07:43.064788 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062017 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 13:07:43.064788 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062020 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 13:07:43.064788 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062022 2531 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 13:07:43.064788 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062025 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 13:07:43.064788 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062027 2531 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 13:07:43.064788 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062030 2531 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 13:07:43.064788 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062032 2531 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 13:07:43.064788 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062034 2531 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 13:07:43.064788 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062037 2531 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 13:07:43.064788 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062046 2531 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 13:07:43.064788 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062049 2531 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 13:07:43.064788 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062051 2531 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 13:07:43.065251 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062056 2531 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 13:07:43.065251 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062059 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 13:07:43.065251 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062062 2531 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 13:07:43.065251 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062065 2531 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 13:07:43.065251 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062067 2531 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 13:07:43.065251 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062070 2531 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 13:07:43.065251 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062073 2531 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 13:07:43.065251 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062076 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 13:07:43.065251 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062079 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 13:07:43.065251 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062082 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 13:07:43.065251 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062085 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 13:07:43.065251 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062087 2531 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 13:07:43.065251 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062090 2531 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 13:07:43.065251 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062092 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 13:07:43.065251 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062095 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 13:07:43.065251 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062098 2531 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 13:07:43.065251 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062100 2531 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 13:07:43.065251 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062103 2531 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 13:07:43.065251 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062106 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 13:07:43.065251 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062109 2531 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 13:07:43.065856 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062111 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 13:07:43.065856 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062114 2531 feature_gate.go:328] unrecognized feature gate: Example Jul 23 13:07:43.065856 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062541 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 13:07:43.065856 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062546 2531 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 13:07:43.065856 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062549 2531 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 13:07:43.065856 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062551 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 13:07:43.065856 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062554 2531 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 13:07:43.065856 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062557 2531 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 13:07:43.065856 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062560 2531 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 13:07:43.065856 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062564 2531 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 13:07:43.065856 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062568 2531 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 13:07:43.065856 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062571 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 13:07:43.065856 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062574 2531 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 13:07:43.065856 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062576 2531 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 13:07:43.065856 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062580 2531 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 13:07:43.065856 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062582 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 13:07:43.065856 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062585 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 13:07:43.065856 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062587 2531 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 13:07:43.065856 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062590 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 13:07:43.065856 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062592 2531 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 13:07:43.066348 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062595 2531 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 13:07:43.066348 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062598 2531 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 13:07:43.066348 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062600 2531 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 13:07:43.066348 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062602 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 13:07:43.066348 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062605 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 13:07:43.066348 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062607 2531 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 13:07:43.066348 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062609 2531 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 13:07:43.066348 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062612 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 13:07:43.066348 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062614 2531 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 13:07:43.066348 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062617 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 13:07:43.066348 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062619 2531 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 13:07:43.066348 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062621 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 13:07:43.066348 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062624 2531 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 13:07:43.066348 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062626 2531 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 13:07:43.066348 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062629 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 13:07:43.066348 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062632 2531 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 13:07:43.066348 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062634 2531 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 13:07:43.066348 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062636 2531 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 13:07:43.066348 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062639 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 13:07:43.066820 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062641 2531 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 13:07:43.066820 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062644 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 13:07:43.066820 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062647 2531 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 13:07:43.066820 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062650 2531 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 13:07:43.066820 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062673 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 13:07:43.066820 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062676 2531 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 13:07:43.066820 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062679 2531 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 13:07:43.066820 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062682 2531 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 13:07:43.066820 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062685 2531 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 13:07:43.066820 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062687 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 13:07:43.066820 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062690 2531 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 13:07:43.066820 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062692 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 13:07:43.066820 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062695 2531 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 13:07:43.066820 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062697 2531 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 13:07:43.066820 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062700 2531 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 13:07:43.066820 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062704 2531 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 13:07:43.066820 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062708 2531 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 13:07:43.066820 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062711 2531 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 13:07:43.066820 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062714 2531 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 13:07:43.066820 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062716 2531 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 13:07:43.067297 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062719 2531 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 13:07:43.067297 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062721 2531 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 13:07:43.067297 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062724 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 13:07:43.067297 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062727 2531 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 13:07:43.067297 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062729 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 13:07:43.067297 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062731 2531 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 13:07:43.067297 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062734 2531 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 13:07:43.067297 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062736 2531 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 13:07:43.067297 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062739 2531 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 13:07:43.067297 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062741 2531 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 13:07:43.067297 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062744 2531 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 13:07:43.067297 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062747 2531 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 13:07:43.067297 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062749 2531 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 13:07:43.067297 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062752 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 13:07:43.067297 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062755 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 13:07:43.067297 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062758 2531 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 13:07:43.067297 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062762 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 13:07:43.067297 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062765 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 13:07:43.067297 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062768 2531 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 13:07:43.067297 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062770 2531 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 13:07:43.068171 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062772 2531 feature_gate.go:328] unrecognized feature gate: Example Jul 23 13:07:43.068171 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062775 2531 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 13:07:43.068171 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062779 2531 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 13:07:43.068171 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062782 2531 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 13:07:43.068171 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062784 2531 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 13:07:43.068171 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062787 2531 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 13:07:43.068171 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062790 2531 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 13:07:43.068171 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062792 2531 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 13:07:43.068171 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062795 2531 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 13:07:43.068171 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062797 2531 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 13:07:43.068171 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062800 2531 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 13:07:43.068171 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062803 2531 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 13:07:43.068171 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062805 2531 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 13:07:43.068171 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062807 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 13:07:43.068171 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062810 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 13:07:43.068171 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062813 2531 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 13:07:43.068171 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062815 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 13:07:43.068171 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062817 2531 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 13:07:43.068171 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062820 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 13:07:43.068171 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062822 2531 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 13:07:43.068990 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062824 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 13:07:43.068990 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.062827 2531 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 13:07:43.068990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.062905 2531 flags.go:64] FLAG: --address="0.0.0.0" Jul 23 13:07:43.068990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.062916 2531 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 23 13:07:43.068990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.062925 2531 flags.go:64] FLAG: --anonymous-auth="true" Jul 23 13:07:43.068990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.062929 2531 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 23 13:07:43.068990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.062936 2531 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 23 13:07:43.068990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.062941 2531 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 23 13:07:43.068990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.062949 2531 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 23 13:07:43.068990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.062955 2531 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 23 13:07:43.068990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.062959 2531 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 23 13:07:43.068990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.062962 2531 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 23 13:07:43.068990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.062966 2531 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 23 13:07:43.068990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.062971 2531 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 23 13:07:43.068990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.062976 2531 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 23 13:07:43.068990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.062979 2531 flags.go:64] FLAG: --cgroup-root="" Jul 23 13:07:43.068990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.062982 2531 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 23 13:07:43.068990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.062984 2531 flags.go:64] FLAG: --client-ca-file="" Jul 23 13:07:43.068990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.062987 2531 flags.go:64] FLAG: --cloud-provider="external" Jul 23 13:07:43.068990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.062990 2531 flags.go:64] FLAG: --cluster-dns="[]" Jul 23 13:07:43.068990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.062994 2531 flags.go:64] FLAG: --cluster-domain="" Jul 23 13:07:43.068990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.062997 2531 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 23 13:07:43.068990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063000 2531 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 23 13:07:43.069981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063003 2531 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 23 13:07:43.069981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063006 2531 flags.go:64] FLAG: --container-log-max-files="5" Jul 23 13:07:43.069981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063010 2531 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 23 13:07:43.069981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063013 2531 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 23 13:07:43.069981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063016 2531 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 23 13:07:43.069981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063020 2531 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 23 13:07:43.069981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063022 2531 flags.go:64] FLAG: --contention-profiling="false" Jul 23 13:07:43.069981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063025 2531 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 23 13:07:43.069981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063028 2531 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 23 13:07:43.069981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063031 2531 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 23 13:07:43.069981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063034 2531 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 23 13:07:43.069981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063038 2531 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 23 13:07:43.069981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063040 2531 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 23 13:07:43.069981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063043 2531 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 23 13:07:43.069981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063046 2531 flags.go:64] FLAG: --enable-load-reader="false" Jul 23 13:07:43.069981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063049 2531 flags.go:64] FLAG: --enable-server="true" Jul 23 13:07:43.069981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063052 2531 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 23 13:07:43.069981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063059 2531 flags.go:64] FLAG: --event-burst="100" Jul 23 13:07:43.069981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063062 2531 flags.go:64] FLAG: --event-qps="50" Jul 23 13:07:43.069981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063065 2531 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 23 13:07:43.069981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063069 2531 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 23 13:07:43.069981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063071 2531 flags.go:64] FLAG: --eviction-hard="" Jul 23 13:07:43.069981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063075 2531 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 23 13:07:43.069981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063078 2531 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 23 13:07:43.070766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063081 2531 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 23 13:07:43.070766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063084 2531 flags.go:64] FLAG: --eviction-soft="" Jul 23 13:07:43.070766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063086 2531 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 23 13:07:43.070766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063089 2531 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 23 13:07:43.070766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063092 2531 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 23 13:07:43.070766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063095 2531 flags.go:64] FLAG: --experimental-mounter-path="" Jul 23 13:07:43.070766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063097 2531 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 23 13:07:43.070766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063100 2531 flags.go:64] FLAG: --fail-swap-on="true" Jul 23 13:07:43.070766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063103 2531 flags.go:64] FLAG: --feature-gates="" Jul 23 13:07:43.070766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063106 2531 flags.go:64] FLAG: --file-check-frequency="20s" Jul 23 13:07:43.070766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063109 2531 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 23 13:07:43.070766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063113 2531 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 23 13:07:43.070766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063116 2531 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 23 13:07:43.070766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063118 2531 flags.go:64] FLAG: --healthz-port="10248" Jul 23 13:07:43.070766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063121 2531 flags.go:64] FLAG: --help="false" Jul 23 13:07:43.070766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063124 2531 flags.go:64] FLAG: --hostname-override="ip-10-0-130-1.ec2.internal" Jul 23 13:07:43.070766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063127 2531 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 23 13:07:43.070766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063130 2531 flags.go:64] FLAG: --http-check-frequency="20s" Jul 23 13:07:43.070766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063133 2531 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 23 13:07:43.070766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063136 2531 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 23 13:07:43.070766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063139 2531 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 23 13:07:43.070766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063142 2531 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 23 13:07:43.070766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063144 2531 flags.go:64] FLAG: --image-service-endpoint="" Jul 23 13:07:43.071381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063148 2531 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 23 13:07:43.071381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063150 2531 flags.go:64] FLAG: --kube-api-burst="100" Jul 23 13:07:43.071381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063156 2531 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 23 13:07:43.071381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063159 2531 flags.go:64] FLAG: --kube-api-qps="50" Jul 23 13:07:43.071381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063162 2531 flags.go:64] FLAG: --kube-reserved="" Jul 23 13:07:43.071381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063165 2531 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 23 13:07:43.071381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063169 2531 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 23 13:07:43.071381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063172 2531 flags.go:64] FLAG: --kubelet-cgroups="" Jul 23 13:07:43.071381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063175 2531 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 23 13:07:43.071381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063178 2531 flags.go:64] FLAG: --lock-file="" Jul 23 13:07:43.071381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063181 2531 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 23 13:07:43.071381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063183 2531 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 23 13:07:43.071381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063186 2531 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 23 13:07:43.071381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063192 2531 flags.go:64] FLAG: --log-json-split-stream="false" Jul 23 13:07:43.071381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063195 2531 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 23 13:07:43.071381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063198 2531 flags.go:64] FLAG: --log-text-split-stream="false" Jul 23 13:07:43.071381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063201 2531 flags.go:64] FLAG: --logging-format="text" Jul 23 13:07:43.071381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063204 2531 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 23 13:07:43.071381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063207 2531 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 23 13:07:43.071381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063210 2531 flags.go:64] FLAG: --manifest-url="" Jul 23 13:07:43.071381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063212 2531 flags.go:64] FLAG: --manifest-url-header="" Jul 23 13:07:43.071381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063217 2531 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 23 13:07:43.071381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063220 2531 flags.go:64] FLAG: --max-open-files="1000000" Jul 23 13:07:43.071381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063224 2531 flags.go:64] FLAG: --max-pods="110" Jul 23 13:07:43.071381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063227 2531 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 23 13:07:43.071986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063230 2531 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 23 13:07:43.071986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063233 2531 flags.go:64] FLAG: --memory-manager-policy="None" Jul 23 13:07:43.071986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063236 2531 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 23 13:07:43.071986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063239 2531 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 23 13:07:43.071986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063242 2531 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 23 13:07:43.071986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063244 2531 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 23 13:07:43.071986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063251 2531 flags.go:64] FLAG: --node-status-max-images="50" Jul 23 13:07:43.071986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063254 2531 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 23 13:07:43.071986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063257 2531 flags.go:64] FLAG: --oom-score-adj="-999" Jul 23 13:07:43.071986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063261 2531 flags.go:64] FLAG: --pod-cidr="" Jul 23 13:07:43.071986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063263 2531 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 23 13:07:43.071986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063268 2531 flags.go:64] FLAG: --pod-manifest-path="" Jul 23 13:07:43.071986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063271 2531 flags.go:64] FLAG: --pod-max-pids="-1" Jul 23 13:07:43.071986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063274 2531 flags.go:64] FLAG: --pods-per-core="0" Jul 23 13:07:43.071986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063277 2531 flags.go:64] FLAG: --port="10250" Jul 23 13:07:43.071986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063280 2531 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 23 13:07:43.071986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063283 2531 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-059d7c31bc46a5bac" Jul 23 13:07:43.071986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063286 2531 flags.go:64] FLAG: --qos-reserved="" Jul 23 13:07:43.071986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063289 2531 flags.go:64] FLAG: --read-only-port="10255" Jul 23 13:07:43.071986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063292 2531 flags.go:64] FLAG: --register-node="true" Jul 23 13:07:43.071986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063295 2531 flags.go:64] FLAG: --register-with-taints="" Jul 23 13:07:43.071986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063298 2531 flags.go:64] FLAG: --registry-burst="10" Jul 23 13:07:43.071986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063301 2531 flags.go:64] FLAG: --registry-qps="5" Jul 23 13:07:43.071986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063304 2531 flags.go:64] FLAG: --reserved-cpus="" Jul 23 13:07:43.072587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063306 2531 flags.go:64] FLAG: --reserved-memory="" Jul 23 13:07:43.072587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063310 2531 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 23 13:07:43.072587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063313 2531 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 23 13:07:43.072587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063316 2531 flags.go:64] FLAG: --rotate-certificates="false" Jul 23 13:07:43.072587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063318 2531 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 23 13:07:43.072587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063321 2531 flags.go:64] FLAG: --runonce="false" Jul 23 13:07:43.072587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063324 2531 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 23 13:07:43.072587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063327 2531 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 23 13:07:43.072587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063330 2531 flags.go:64] FLAG: --seccomp-default="false" Jul 23 13:07:43.072587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063333 2531 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 23 13:07:43.072587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063335 2531 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 23 13:07:43.072587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063338 2531 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 23 13:07:43.072587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063341 2531 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 23 13:07:43.072587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063344 2531 flags.go:64] FLAG: --storage-driver-password="root" Jul 23 13:07:43.072587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063347 2531 flags.go:64] FLAG: --storage-driver-secure="false" Jul 23 13:07:43.072587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063366 2531 flags.go:64] FLAG: --storage-driver-table="stats" Jul 23 13:07:43.072587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063369 2531 flags.go:64] FLAG: --storage-driver-user="root" Jul 23 13:07:43.072587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063372 2531 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 23 13:07:43.072587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063376 2531 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 23 13:07:43.072587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063379 2531 flags.go:64] FLAG: --system-cgroups="" Jul 23 13:07:43.072587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063382 2531 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 23 13:07:43.072587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063387 2531 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 23 13:07:43.072587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063390 2531 flags.go:64] FLAG: --tls-cert-file="" Jul 23 13:07:43.072587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063393 2531 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 23 13:07:43.072587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063397 2531 flags.go:64] FLAG: --tls-min-version="" Jul 23 13:07:43.073177 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063400 2531 flags.go:64] FLAG: --tls-private-key-file="" Jul 23 13:07:43.073177 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063403 2531 flags.go:64] FLAG: --topology-manager-policy="none" Jul 23 13:07:43.073177 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063406 2531 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 23 13:07:43.073177 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063408 2531 flags.go:64] FLAG: --topology-manager-scope="container" Jul 23 13:07:43.073177 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063411 2531 flags.go:64] FLAG: --v="2" Jul 23 13:07:43.073177 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063415 2531 flags.go:64] FLAG: --version="false" Jul 23 13:07:43.073177 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063419 2531 flags.go:64] FLAG: --vmodule="" Jul 23 13:07:43.073177 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063423 2531 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 23 13:07:43.073177 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.063426 2531 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 23 13:07:43.073177 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063542 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 13:07:43.073177 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063545 2531 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 13:07:43.073177 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063548 2531 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 13:07:43.073177 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063551 2531 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 13:07:43.073177 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063553 2531 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 13:07:43.073177 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063555 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 13:07:43.073177 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063558 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 13:07:43.073177 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063561 2531 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 13:07:43.073177 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063563 2531 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 13:07:43.073177 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063566 2531 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 13:07:43.073177 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063569 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 13:07:43.073177 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063571 2531 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 13:07:43.073177 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063574 2531 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 13:07:43.073770 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063578 2531 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 13:07:43.073770 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063580 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 13:07:43.073770 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063583 2531 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 13:07:43.073770 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063586 2531 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 13:07:43.073770 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063589 2531 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 13:07:43.073770 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063592 2531 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 13:07:43.073770 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063595 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 13:07:43.073770 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063598 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 13:07:43.073770 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063600 2531 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 13:07:43.073770 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063603 2531 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 13:07:43.073770 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063605 2531 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 13:07:43.073770 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063608 2531 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 13:07:43.073770 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063610 2531 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 13:07:43.073770 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063613 2531 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 13:07:43.073770 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063615 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 13:07:43.073770 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063618 2531 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 13:07:43.073770 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063621 2531 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 13:07:43.073770 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063623 2531 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 13:07:43.073770 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063626 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 13:07:43.074233 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063629 2531 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 13:07:43.074233 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063631 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 13:07:43.074233 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063634 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 13:07:43.074233 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063636 2531 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 13:07:43.074233 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063639 2531 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 13:07:43.074233 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063641 2531 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 13:07:43.074233 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063643 2531 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 13:07:43.074233 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063646 2531 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 13:07:43.074233 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063648 2531 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 13:07:43.074233 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063651 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 13:07:43.074233 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063653 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 13:07:43.074233 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063655 2531 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 13:07:43.074233 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063658 2531 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 13:07:43.074233 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063661 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 13:07:43.074233 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063663 2531 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 13:07:43.074233 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063666 2531 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 13:07:43.074233 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063670 2531 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 13:07:43.074233 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063673 2531 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 13:07:43.074233 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063675 2531 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 13:07:43.074233 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063678 2531 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 13:07:43.074731 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063680 2531 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 13:07:43.074731 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063683 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 13:07:43.074731 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063685 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 13:07:43.074731 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063688 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 13:07:43.074731 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063690 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 13:07:43.074731 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063693 2531 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 13:07:43.074731 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063695 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 13:07:43.074731 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063697 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 13:07:43.074731 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063700 2531 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 13:07:43.074731 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063702 2531 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 13:07:43.074731 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063705 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 13:07:43.074731 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063707 2531 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 13:07:43.074731 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063709 2531 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 13:07:43.074731 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063712 2531 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 13:07:43.074731 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063714 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 13:07:43.074731 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063716 2531 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 13:07:43.074731 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063719 2531 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 13:07:43.074731 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063721 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 13:07:43.074731 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063725 2531 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 13:07:43.075195 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063728 2531 feature_gate.go:328] unrecognized feature gate: Example Jul 23 13:07:43.075195 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063731 2531 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 13:07:43.075195 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063734 2531 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 13:07:43.075195 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063737 2531 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 13:07:43.075195 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063739 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 13:07:43.075195 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063742 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 13:07:43.075195 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063746 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 13:07:43.075195 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063749 2531 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 13:07:43.075195 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063752 2531 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 13:07:43.075195 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063756 2531 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 13:07:43.075195 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063759 2531 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 13:07:43.075195 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063762 2531 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 13:07:43.075195 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063766 2531 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 13:07:43.075195 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063768 2531 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 13:07:43.075195 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063771 2531 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 13:07:43.075195 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063774 2531 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 13:07:43.075195 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063777 2531 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 13:07:43.075195 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063780 2531 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 13:07:43.075195 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063782 2531 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 13:07:43.075195 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063785 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 13:07:43.075812 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063788 2531 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 13:07:43.075812 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063790 2531 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 13:07:43.075812 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063793 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 13:07:43.075812 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063795 2531 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 13:07:43.075812 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063797 2531 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 13:07:43.075812 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063801 2531 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 13:07:43.075812 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063805 2531 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 13:07:43.075812 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.063807 2531 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 13:07:43.075812 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.064338 2531 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 13:07:43.075812 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.072329 2531 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 23 13:07:43.075812 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.072345 2531 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 23 13:07:43.075812 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.073120 2531 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 23 13:07:43.075812 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.073131 2531 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 23 13:07:43.075812 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073193 2531 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 13:07:43.075812 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073199 2531 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 13:07:43.076185 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073210 2531 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 13:07:43.076185 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073214 2531 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 13:07:43.076185 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073216 2531 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 13:07:43.076185 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073219 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 13:07:43.076185 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073222 2531 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 13:07:43.076185 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073224 2531 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 13:07:43.076185 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073227 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 13:07:43.076185 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073230 2531 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 13:07:43.076185 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073232 2531 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 13:07:43.076185 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073235 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 13:07:43.076185 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073238 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 13:07:43.076185 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073240 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 13:07:43.076185 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073243 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 13:07:43.076185 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073245 2531 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 13:07:43.076185 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073252 2531 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 13:07:43.076185 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073254 2531 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 13:07:43.076185 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073257 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 13:07:43.076185 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073259 2531 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 13:07:43.076185 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073261 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 13:07:43.076185 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073264 2531 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 13:07:43.076730 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073267 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 13:07:43.076730 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073269 2531 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 13:07:43.076730 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073272 2531 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 13:07:43.076730 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073274 2531 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 13:07:43.076730 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073277 2531 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 13:07:43.076730 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073279 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 13:07:43.076730 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073283 2531 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 13:07:43.076730 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073288 2531 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 13:07:43.076730 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073291 2531 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 13:07:43.076730 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073294 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 13:07:43.076730 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073297 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 13:07:43.076730 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073299 2531 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 13:07:43.076730 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073302 2531 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 13:07:43.076730 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073305 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 13:07:43.076730 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073307 2531 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 13:07:43.076730 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073310 2531 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 13:07:43.076730 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073312 2531 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 13:07:43.076730 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073315 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 13:07:43.076730 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073317 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 13:07:43.076730 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073319 2531 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 13:07:43.077262 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073322 2531 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 13:07:43.077262 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073324 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 13:07:43.077262 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073327 2531 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 13:07:43.077262 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073329 2531 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 13:07:43.077262 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073332 2531 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 13:07:43.077262 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073335 2531 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 13:07:43.077262 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073338 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 13:07:43.077262 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073340 2531 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 13:07:43.077262 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073343 2531 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 13:07:43.077262 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073346 2531 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 13:07:43.077262 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073348 2531 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 13:07:43.077262 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073366 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 13:07:43.077262 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073369 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 13:07:43.077262 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073371 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 13:07:43.077262 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073374 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 13:07:43.077262 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073377 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 13:07:43.077262 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073379 2531 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 13:07:43.077262 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073382 2531 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 13:07:43.077262 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073385 2531 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 13:07:43.077262 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073388 2531 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 13:07:43.077792 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073391 2531 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 13:07:43.077792 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073394 2531 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 13:07:43.077792 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073396 2531 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 13:07:43.077792 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073399 2531 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 13:07:43.077792 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073402 2531 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 13:07:43.077792 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073404 2531 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 13:07:43.077792 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073407 2531 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 13:07:43.077792 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073409 2531 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 13:07:43.077792 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073412 2531 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 13:07:43.077792 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073415 2531 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 13:07:43.077792 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073418 2531 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 13:07:43.077792 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073422 2531 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 13:07:43.077792 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073425 2531 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 13:07:43.077792 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073428 2531 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 13:07:43.077792 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073431 2531 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 13:07:43.077792 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073434 2531 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 13:07:43.077792 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073437 2531 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 13:07:43.077792 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073439 2531 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 13:07:43.077792 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073442 2531 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 13:07:43.077792 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073444 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 13:07:43.078283 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073447 2531 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 13:07:43.078283 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073450 2531 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 13:07:43.078283 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073452 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 13:07:43.078283 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073455 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 13:07:43.078283 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073457 2531 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 13:07:43.078283 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073460 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 13:07:43.078283 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073464 2531 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 13:07:43.078283 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073467 2531 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 13:07:43.078283 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073470 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 13:07:43.078283 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073473 2531 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 13:07:43.078283 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073475 2531 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 13:07:43.078283 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073478 2531 feature_gate.go:328] unrecognized feature gate: Example Jul 23 13:07:43.078283 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073480 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 13:07:43.078283 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073483 2531 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 13:07:43.078283 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073485 2531 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 13:07:43.078283 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073488 2531 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 13:07:43.078283 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073490 2531 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 13:07:43.078720 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.073495 2531 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 13:07:43.078720 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073620 2531 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 13:07:43.078720 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073627 2531 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 13:07:43.078720 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073630 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 13:07:43.078720 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073633 2531 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 13:07:43.078720 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073636 2531 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 13:07:43.078720 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073639 2531 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 13:07:43.078720 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073642 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 13:07:43.078720 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073645 2531 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 13:07:43.078720 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073648 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 13:07:43.078720 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073650 2531 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 13:07:43.078720 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073653 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 13:07:43.078720 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073655 2531 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 13:07:43.078720 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073658 2531 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 13:07:43.078720 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073660 2531 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 13:07:43.079104 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073664 2531 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 13:07:43.079104 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073669 2531 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 13:07:43.079104 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073672 2531 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 13:07:43.079104 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073675 2531 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 13:07:43.079104 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073678 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 13:07:43.079104 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073680 2531 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 13:07:43.079104 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073683 2531 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 13:07:43.079104 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073686 2531 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 13:07:43.079104 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073688 2531 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 13:07:43.079104 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073691 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 13:07:43.079104 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073693 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 13:07:43.079104 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073696 2531 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 13:07:43.079104 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073698 2531 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 13:07:43.079104 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073701 2531 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 13:07:43.079104 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073703 2531 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 13:07:43.079104 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073705 2531 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 13:07:43.079104 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073708 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 13:07:43.079104 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073711 2531 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 13:07:43.079104 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073713 2531 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 13:07:43.079635 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073716 2531 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 13:07:43.079635 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073719 2531 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 13:07:43.079635 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073722 2531 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 13:07:43.079635 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073726 2531 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 13:07:43.079635 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073729 2531 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 13:07:43.079635 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073732 2531 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 13:07:43.079635 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073735 2531 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 13:07:43.079635 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073738 2531 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 13:07:43.079635 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073740 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 13:07:43.079635 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073743 2531 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 13:07:43.079635 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073746 2531 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 13:07:43.079635 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073749 2531 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 13:07:43.079635 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073751 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 13:07:43.079635 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073754 2531 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 13:07:43.079635 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073757 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 13:07:43.079635 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073760 2531 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 13:07:43.079635 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073763 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 13:07:43.079635 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073765 2531 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 13:07:43.079635 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073767 2531 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 13:07:43.080091 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073770 2531 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 13:07:43.080091 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073773 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 13:07:43.080091 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073776 2531 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 13:07:43.080091 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073778 2531 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 13:07:43.080091 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073781 2531 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 13:07:43.080091 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073783 2531 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 13:07:43.080091 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073786 2531 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 13:07:43.080091 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073788 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 13:07:43.080091 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073791 2531 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 13:07:43.080091 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073793 2531 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 13:07:43.080091 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073795 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 13:07:43.080091 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073798 2531 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 13:07:43.080091 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073800 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 13:07:43.080091 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073803 2531 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 13:07:43.080091 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073805 2531 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 13:07:43.080091 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073808 2531 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 13:07:43.080091 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073810 2531 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 13:07:43.080091 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073812 2531 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 13:07:43.080091 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073815 2531 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 13:07:43.080091 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073818 2531 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 13:07:43.080775 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073820 2531 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 13:07:43.080775 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073822 2531 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 13:07:43.080775 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073825 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 13:07:43.080775 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073827 2531 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 13:07:43.080775 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073829 2531 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 13:07:43.080775 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073832 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 13:07:43.080775 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073835 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 13:07:43.080775 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073837 2531 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 13:07:43.080775 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073839 2531 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 13:07:43.080775 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073842 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 13:07:43.080775 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073844 2531 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 13:07:43.080775 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073847 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 13:07:43.080775 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073849 2531 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 13:07:43.080775 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073852 2531 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 13:07:43.080775 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073855 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 13:07:43.080775 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073857 2531 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 13:07:43.080775 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073860 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 13:07:43.080775 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073862 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 13:07:43.080775 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073865 2531 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 13:07:43.080775 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073867 2531 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 13:07:43.081584 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073870 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 13:07:43.081584 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073872 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 13:07:43.081584 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073875 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 13:07:43.081584 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073877 2531 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 13:07:43.081584 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073880 2531 feature_gate.go:328] unrecognized feature gate: Example Jul 23 13:07:43.081584 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073882 2531 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 13:07:43.081584 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:43.073884 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 13:07:43.081584 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.073890 2531 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 13:07:43.081584 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.074005 2531 server.go:962] "Client rotation is on, will bootstrap in background" Jul 23 13:07:43.081584 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.076241 2531 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 23 13:07:43.081584 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.077940 2531 server.go:1019] "Starting client certificate rotation" Jul 23 13:07:43.081584 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.078034 2531 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 13:07:43.081584 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.078067 2531 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 13:07:43.082166 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.082147 2531 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 13:07:43.086038 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.086015 2531 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 13:07:43.096915 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.096896 2531 log.go:25] "Validated CRI v1 runtime API" Jul 23 13:07:43.099191 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.099177 2531 log.go:25] "Validated CRI v1 image API" Jul 23 13:07:43.100016 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.100002 2531 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 23 13:07:43.104174 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.104155 2531 fs.go:135] Filesystem UUIDs: map[6d9e7d8d-9f4a-4eba-a92e-067583c321a6:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f70135ae-bb87-4462-b489-0c263f619caf:/dev/nvme0n1p4] Jul 23 13:07:43.104235 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.104173 2531 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 23 13:07:43.105888 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.105871 2531 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 13:07:43.109242 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.109126 2531 manager.go:216] Machine: {Timestamp:2026-07-23 13:07:43.107865599 +0000 UTC m=+0.347141566 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099389 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d53bc28bef7cc36a5d1276c98b2a9 SystemUUID:ec2d53bc-28be-f7cc-36a5-d1276c98b2a9 BootID:4d830b6d-54b9-4db3-90ab-6499406cd123 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d5:81:d7:db:23 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d5:81:d7:db:23 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e2:51:57:68:82:a6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 23 13:07:43.109242 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.109239 2531 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 23 13:07:43.109368 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.109321 2531 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 23 13:07:43.111295 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.111273 2531 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 23 13:07:43.111451 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.111301 2531 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-1.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 23 13:07:43.112634 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.112622 2531 topology_manager.go:138] "Creating topology manager with none policy" Jul 23 13:07:43.112672 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.112637 2531 container_manager_linux.go:309] "Creating device plugin manager" Jul 23 13:07:43.112672 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.112649 2531 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 13:07:43.113263 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.113252 2531 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 13:07:43.114761 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.114751 2531 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 23 13:07:43.118116 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.118105 2531 state_mem.go:36] "Initialized new in-memory state store" Jul 23 13:07:43.118230 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.118221 2531 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 23 13:07:43.118266 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.118251 2531 kubelet.go:486] "Attempting to sync node with API server" Jul 23 13:07:43.118316 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.118268 2531 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 23 13:07:43.118316 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.118286 2531 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 23 13:07:43.118316 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.118296 2531 kubelet.go:398] "Adding apiserver pod source" Jul 23 13:07:43.118316 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.118307 2531 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 23 13:07:43.118690 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.118677 2531 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 13:07:43.118748 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.118697 2531 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 13:07:43.121307 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.121289 2531 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 23 13:07:43.124148 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.124013 2531 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 23 13:07:43.124148 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.124062 2531 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 23 13:07:43.125196 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.125177 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 23 13:07:43.125299 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.125210 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 23 13:07:43.125299 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.125218 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 23 13:07:43.125299 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.125224 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 23 13:07:43.125299 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.125238 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 23 13:07:43.125299 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.125243 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 23 13:07:43.125299 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.125254 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 23 13:07:43.125299 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.125262 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 23 13:07:43.125299 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.125273 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 23 13:07:43.125299 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.125282 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 23 13:07:43.125299 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.125294 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 23 13:07:43.125299 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.125302 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 23 13:07:43.126079 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.126061 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 23 13:07:43.126127 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.126095 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 23 13:07:43.127319 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.127293 2531 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f2fc8" Jul 23 13:07:43.129229 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.129199 2531 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-1.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 23 13:07:43.129754 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.129738 2531 server.go:1268] "Started kubelet" Jul 23 13:07:43.129823 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:43.129796 2531 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 23 13:07:43.129923 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.129829 2531 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 23 13:07:43.129923 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:43.129860 2531 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-1.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 23 13:07:43.129923 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.129858 2531 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 23 13:07:43.130034 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.129941 2531 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 23 13:07:43.130704 ip-10-0-130-1 systemd[1]: Started Kubernetes Kubelet. Jul 23 13:07:43.132320 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.132306 2531 server.go:310] "Adding debug handlers to kubelet server" Jul 23 13:07:43.132896 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.132879 2531 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 23 13:07:43.135499 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.135472 2531 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f2fc8" Jul 23 13:07:43.136003 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:43.135987 2531 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 23 13:07:43.136716 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:43.135590 2531 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-1.ec2.internal.18c4ed1d75ad2aac default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-1.ec2.internal,UID:ip-10-0-130-1.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-1.ec2.internal,},FirstTimestamp:2026-07-23 13:07:43.12967646 +0000 UTC m=+0.368952428,LastTimestamp:2026-07-23 13:07:43.12967646 +0000 UTC m=+0.368952428,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-1.ec2.internal,}" Jul 23 13:07:43.141720 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.141700 2531 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 23 13:07:43.141720 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.141717 2531 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 23 13:07:43.142065 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:43.142046 2531 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-1.ec2.internal\" not found" Jul 23 13:07:43.142268 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.142250 2531 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 23 13:07:43.142343 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.142264 2531 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 23 13:07:43.142343 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.142269 2531 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 23 13:07:43.142466 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.142451 2531 reconstruct.go:97] "Volume reconstruction finished" Jul 23 13:07:43.142466 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.142462 2531 factory.go:55] Registering systemd factory Jul 23 13:07:43.142551 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.142476 2531 factory.go:223] Registration of the systemd container factory successfully Jul 23 13:07:43.142551 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.142467 2531 reconciler.go:29] "Reconciler: start to sync state" Jul 23 13:07:43.142742 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.142727 2531 factory.go:153] Registering CRI-O factory Jul 23 13:07:43.142742 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.142743 2531 factory.go:223] Registration of the crio container factory successfully Jul 23 13:07:43.142884 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.142807 2531 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 23 13:07:43.142884 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.142832 2531 factory.go:103] Registering Raw factory Jul 23 13:07:43.142884 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.142848 2531 manager.go:1196] Started watching for new ooms in manager Jul 23 13:07:43.143401 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.143384 2531 manager.go:349] Starting recovery of all containers Jul 23 13:07:43.154469 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.154290 2531 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 13:07:43.154672 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.154652 2531 manager.go:354] Recovery completed Jul 23 13:07:43.157115 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:43.157078 2531 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-130-1.ec2.internal\" not found" node="ip-10-0-130-1.ec2.internal" Jul 23 13:07:43.160427 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.160414 2531 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 13:07:43.162124 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.162109 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-1.ec2.internal" event="NodeHasSufficientMemory" Jul 23 13:07:43.162175 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.162139 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-1.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 13:07:43.162175 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.162153 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-1.ec2.internal" event="NodeHasSufficientPID" Jul 23 13:07:43.162660 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.162646 2531 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 23 13:07:43.162735 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.162658 2531 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 23 13:07:43.162735 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.162663 2531 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 13:07:43.162735 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.162678 2531 state_mem.go:36] "Initialized new in-memory state store" Jul 23 13:07:43.163297 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.163283 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-1.ec2.internal" event="NodeHasSufficientMemory" Jul 23 13:07:43.163346 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.163309 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-1.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 13:07:43.163346 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.163322 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-1.ec2.internal" event="NodeHasSufficientPID" Jul 23 13:07:43.163983 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.163971 2531 policy_none.go:49] "None policy: Start" Jul 23 13:07:43.164048 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.163995 2531 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 23 13:07:43.164048 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.164008 2531 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 23 13:07:43.165193 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.165182 2531 policy_none.go:47] "Start" Jul 23 13:07:43.204093 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.204077 2531 manager.go:337] "Starting Device Plugin manager" Jul 23 13:07:43.204203 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:43.204106 2531 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 23 13:07:43.204203 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.204115 2531 server.go:87] "Starting device plugin registration server" Jul 23 13:07:43.204399 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.204387 2531 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 23 13:07:43.204435 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.204403 2531 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 23 13:07:43.204574 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.204500 2531 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 23 13:07:43.204664 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.204589 2531 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 23 13:07:43.204664 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.204600 2531 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 23 13:07:43.205089 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:43.205073 2531 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 23 13:07:43.205159 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:43.205129 2531 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-1.ec2.internal\" not found" Jul 23 13:07:43.304576 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.304506 2531 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 13:07:43.305826 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.305442 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-1.ec2.internal" event="NodeHasSufficientMemory" Jul 23 13:07:43.305826 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.305470 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-1.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 13:07:43.305826 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.305479 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-1.ec2.internal" event="NodeHasSufficientPID" Jul 23 13:07:43.305826 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.305506 2531 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-1.ec2.internal" Jul 23 13:07:43.311483 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.311455 2531 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-1.ec2.internal" Jul 23 13:07:43.311585 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:43.311483 2531 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-1.ec2.internal\": node \"ip-10-0-130-1.ec2.internal\" not found" Jul 23 13:07:43.324335 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.324312 2531 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 23 13:07:43.325689 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.325667 2531 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 23 13:07:43.325778 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.325702 2531 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 23 13:07:43.325778 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.325722 2531 kubelet.go:2448] "Starting kubelet main sync loop" Jul 23 13:07:43.325778 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:43.325756 2531 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 23 13:07:43.328214 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.328193 2531 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 13:07:43.329492 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:43.329477 2531 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-1.ec2.internal\" not found" Jul 23 13:07:43.426278 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.426244 2531 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-1.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-1.ec2.internal"] Jul 23 13:07:43.426431 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.426321 2531 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 13:07:43.427989 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.427976 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-1.ec2.internal" event="NodeHasSufficientMemory" Jul 23 13:07:43.428056 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.428004 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-1.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 13:07:43.428056 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.428014 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-1.ec2.internal" event="NodeHasSufficientPID" Jul 23 13:07:43.429335 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.429322 2531 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 13:07:43.429494 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.429478 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-1.ec2.internal" Jul 23 13:07:43.429552 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.429522 2531 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 13:07:43.429604 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:43.429557 2531 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-1.ec2.internal\" not found" Jul 23 13:07:43.430070 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.430051 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-1.ec2.internal" event="NodeHasSufficientMemory" Jul 23 13:07:43.430141 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.430087 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-1.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 13:07:43.430141 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.430099 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-1.ec2.internal" event="NodeHasSufficientPID" Jul 23 13:07:43.430141 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.430117 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-1.ec2.internal" event="NodeHasSufficientMemory" Jul 23 13:07:43.430141 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.430136 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-1.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 13:07:43.430291 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.430145 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-1.ec2.internal" event="NodeHasSufficientPID" Jul 23 13:07:43.431346 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.431330 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-1.ec2.internal" Jul 23 13:07:43.431425 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.431374 2531 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 13:07:43.432089 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.432074 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-1.ec2.internal" event="NodeHasSufficientMemory" Jul 23 13:07:43.432156 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.432094 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-1.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 13:07:43.432156 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.432105 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-1.ec2.internal" event="NodeHasSufficientPID" Jul 23 13:07:43.444291 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.444275 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e2041f17d49a605f250f8febf662052c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-1.ec2.internal\" (UID: \"e2041f17d49a605f250f8febf662052c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-1.ec2.internal" Jul 23 13:07:43.444342 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.444307 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4b3cd48ad004afdc21879d1579b5d29b-config\") pod \"kube-apiserver-proxy-ip-10-0-130-1.ec2.internal\" (UID: \"4b3cd48ad004afdc21879d1579b5d29b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-1.ec2.internal" Jul 23 13:07:43.444342 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.444331 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e2041f17d49a605f250f8febf662052c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-1.ec2.internal\" (UID: \"e2041f17d49a605f250f8febf662052c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-1.ec2.internal" Jul 23 13:07:43.460060 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:43.460039 2531 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-1.ec2.internal\" not found" node="ip-10-0-130-1.ec2.internal" Jul 23 13:07:43.464442 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:43.464428 2531 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-1.ec2.internal\" not found" node="ip-10-0-130-1.ec2.internal" Jul 23 13:07:43.530486 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:43.530461 2531 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-1.ec2.internal\" not found" Jul 23 13:07:43.544798 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.544774 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e2041f17d49a605f250f8febf662052c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-1.ec2.internal\" (UID: \"e2041f17d49a605f250f8febf662052c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-1.ec2.internal" Jul 23 13:07:43.544852 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.544828 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e2041f17d49a605f250f8febf662052c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-1.ec2.internal\" (UID: \"e2041f17d49a605f250f8febf662052c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-1.ec2.internal" Jul 23 13:07:43.544890 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.544870 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e2041f17d49a605f250f8febf662052c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-1.ec2.internal\" (UID: \"e2041f17d49a605f250f8febf662052c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-1.ec2.internal" Jul 23 13:07:43.544921 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.544896 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4b3cd48ad004afdc21879d1579b5d29b-config\") pod \"kube-apiserver-proxy-ip-10-0-130-1.ec2.internal\" (UID: \"4b3cd48ad004afdc21879d1579b5d29b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-1.ec2.internal" Jul 23 13:07:43.544952 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.544932 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4b3cd48ad004afdc21879d1579b5d29b-config\") pod \"kube-apiserver-proxy-ip-10-0-130-1.ec2.internal\" (UID: \"4b3cd48ad004afdc21879d1579b5d29b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-1.ec2.internal" Jul 23 13:07:43.544983 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.544951 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e2041f17d49a605f250f8febf662052c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-1.ec2.internal\" (UID: \"e2041f17d49a605f250f8febf662052c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-1.ec2.internal" Jul 23 13:07:43.631276 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:43.631210 2531 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-1.ec2.internal\" not found" Jul 23 13:07:43.731757 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:43.731723 2531 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-1.ec2.internal\" not found" Jul 23 13:07:43.763211 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.763188 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-1.ec2.internal" Jul 23 13:07:43.768488 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.768470 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-1.ec2.internal" Jul 23 13:07:43.832832 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:43.832784 2531 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-1.ec2.internal\" not found" Jul 23 13:07:43.933341 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:43.933272 2531 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-1.ec2.internal\" not found" Jul 23 13:07:43.941856 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.941827 2531 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 13:07:43.941969 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.941928 2531 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-1.ec2.internal" Jul 23 13:07:43.951473 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.951453 2531 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 13:07:43.953196 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.953184 2531 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-1.ec2.internal" Jul 23 13:07:43.966908 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:43.966890 2531 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 13:07:44.076628 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.076588 2531 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 23 13:07:44.077109 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.076776 2531 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 13:07:44.077109 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.076786 2531 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 13:07:44.077109 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.076791 2531 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 13:07:44.120743 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.120709 2531 apiserver.go:52] "Watching apiserver" Jul 23 13:07:44.136331 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.136283 2531 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 23 13:07:44.137382 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.137317 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-1.ec2.internal","openshift-multus/multus-additional-cni-plugins-zwp6q","openshift-multus/multus-gckpb","openshift-cluster-node-tuning-operator/tuned-jk4mp","openshift-dns/node-resolver-xlb6j","openshift-image-registry/node-ca-f9q95","openshift-multus/network-metrics-daemon-cp74b","openshift-network-diagnostics/network-check-target-tc68c","openshift-network-operator/iptables-alerter-vbwxm","openshift-ovn-kubernetes/ovnkube-node-t6m8j","kube-system/global-pull-secret-syncer-vfgv8","kube-system/konnectivity-agent-ljjj9","kube-system/kube-apiserver-proxy-ip-10-0-130-1.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj"] Jul 23 13:07:44.139231 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.139194 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ljjj9" Jul 23 13:07:44.140940 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.140899 2531 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 13:02:43 +0000 UTC" deadline="2027-12-20 19:09:27.813868639 +0000 UTC" Jul 23 13:07:44.140940 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.140938 2531 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12366h1m43.67293435s" Jul 23 13:07:44.141810 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.141798 2531 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 23 13:07:44.142006 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.141972 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 23 13:07:44.142132 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.142026 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" Jul 23 13:07:44.142213 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.142169 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-pqzwf\"" Jul 23 13:07:44.142337 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.142320 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 23 13:07:44.143524 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.143506 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:07:44.143623 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:44.143566 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cp74b" podUID="e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3" Jul 23 13:07:44.144075 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.144055 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 23 13:07:44.144168 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.144121 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pxtx2\"" Jul 23 13:07:44.144322 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.144306 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 23 13:07:44.144372 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.144344 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 23 13:07:44.144843 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.144825 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.146161 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.146147 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xlb6j" Jul 23 13:07:44.146977 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.146962 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 23 13:07:44.147048 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.146990 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 23 13:07:44.147103 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.147077 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-22tcb\"" Jul 23 13:07:44.147525 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.147512 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-f9q95" Jul 23 13:07:44.148258 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.148240 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-tq4mm\"" Jul 23 13:07:44.148340 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.148301 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 23 13:07:44.148340 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.148317 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 23 13:07:44.148510 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.148343 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b5ef6285-fd8c-4841-bede-d88d25e6ba2b-agent-certs\") pod \"konnectivity-agent-ljjj9\" (UID: \"b5ef6285-fd8c-4841-bede-d88d25e6ba2b\") " pod="kube-system/konnectivity-agent-ljjj9" Jul 23 13:07:44.148510 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.148372 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b5ef6285-fd8c-4841-bede-d88d25e6ba2b-konnectivity-ca\") pod \"konnectivity-agent-ljjj9\" (UID: \"b5ef6285-fd8c-4841-bede-d88d25e6ba2b\") " pod="kube-system/konnectivity-agent-ljjj9" Jul 23 13:07:44.148510 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.148402 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/04af7f19-0f38-4d43-9b9d-221eb93744c2-socket-dir\") pod \"aws-ebs-csi-driver-node-t27fj\" (UID: \"04af7f19-0f38-4d43-9b9d-221eb93744c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" Jul 23 13:07:44.148510 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.148462 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/04af7f19-0f38-4d43-9b9d-221eb93744c2-registration-dir\") pod \"aws-ebs-csi-driver-node-t27fj\" (UID: \"04af7f19-0f38-4d43-9b9d-221eb93744c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" Jul 23 13:07:44.148510 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.148485 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/04af7f19-0f38-4d43-9b9d-221eb93744c2-device-dir\") pod \"aws-ebs-csi-driver-node-t27fj\" (UID: \"04af7f19-0f38-4d43-9b9d-221eb93744c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" Jul 23 13:07:44.148510 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.148501 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/04af7f19-0f38-4d43-9b9d-221eb93744c2-sys-fs\") pod \"aws-ebs-csi-driver-node-t27fj\" (UID: \"04af7f19-0f38-4d43-9b9d-221eb93744c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" Jul 23 13:07:44.148814 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.148516 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/04af7f19-0f38-4d43-9b9d-221eb93744c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t27fj\" (UID: \"04af7f19-0f38-4d43-9b9d-221eb93744c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" Jul 23 13:07:44.148814 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.148535 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/04af7f19-0f38-4d43-9b9d-221eb93744c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-t27fj\" (UID: \"04af7f19-0f38-4d43-9b9d-221eb93744c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" Jul 23 13:07:44.148814 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.148577 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2lph\" (UniqueName: \"kubernetes.io/projected/04af7f19-0f38-4d43-9b9d-221eb93744c2-kube-api-access-x2lph\") pod \"aws-ebs-csi-driver-node-t27fj\" (UID: \"04af7f19-0f38-4d43-9b9d-221eb93744c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" Jul 23 13:07:44.148814 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.148619 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-metrics-certs\") pod \"network-metrics-daemon-cp74b\" (UID: \"e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3\") " pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:07:44.148814 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.148671 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m2mh\" (UniqueName: \"kubernetes.io/projected/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-kube-api-access-7m2mh\") pod \"network-metrics-daemon-cp74b\" (UID: \"e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3\") " pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:07:44.148986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.148842 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:07:44.148986 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:44.148894 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc68c" podUID="65c6347a-d5cc-4964-b5dd-b5b4098e0c5e" Jul 23 13:07:44.149615 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.149597 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 23 13:07:44.149743 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.149645 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 23 13:07:44.149887 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.149874 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 23 13:07:44.150070 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.150055 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dzzrm\"" Jul 23 13:07:44.150436 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.150420 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vbwxm" Jul 23 13:07:44.152449 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.152433 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.153086 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.153060 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 23 13:07:44.153615 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.153590 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 23 13:07:44.153916 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.153900 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 23 13:07:44.154020 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.153934 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xpt7v\"" Jul 23 13:07:44.154020 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.153940 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:07:44.154132 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:44.154079 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfgv8" podUID="5df5e8b8-2bc3-4788-bf77-4f32a3d52f65" Jul 23 13:07:44.154496 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.154473 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2lrkt\"" Jul 23 13:07:44.154568 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.154479 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 23 13:07:44.154655 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.154621 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 23 13:07:44.154805 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.154791 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 23 13:07:44.154805 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.154799 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 23 13:07:44.155431 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.155408 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.155507 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.155436 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 23 13:07:44.155598 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.155521 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 23 13:07:44.157272 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.157252 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.157582 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.157568 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jw9mv\"" Jul 23 13:07:44.157806 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.157791 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 23 13:07:44.157892 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.157868 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 23 13:07:44.157971 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.157955 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 23 13:07:44.158023 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.157997 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 23 13:07:44.159684 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.159669 2531 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 13:07:44.159966 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.159943 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 23 13:07:44.160060 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.160025 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 23 13:07:44.160060 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.160042 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-l9ft2\"" Jul 23 13:07:44.194221 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.194195 2531 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ccdgk" Jul 23 13:07:44.201003 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.200980 2531 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ccdgk" Jul 23 13:07:44.243296 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.243275 2531 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 23 13:07:44.249324 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.249302 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-etc-modprobe-d\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.249467 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.249330 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-node-log\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.249467 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.249357 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/04af7f19-0f38-4d43-9b9d-221eb93744c2-socket-dir\") pod \"aws-ebs-csi-driver-node-t27fj\" (UID: \"04af7f19-0f38-4d43-9b9d-221eb93744c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" Jul 23 13:07:44.249467 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.249393 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-var-lib-kubelet\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.249467 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.249420 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfzht\" (UniqueName: \"kubernetes.io/projected/dc52fdf9-2a89-4d34-8c45-4ff335a8ca85-kube-api-access-pfzht\") pod \"iptables-alerter-vbwxm\" (UID: \"dc52fdf9-2a89-4d34-8c45-4ff335a8ca85\") " pod="openshift-network-operator/iptables-alerter-vbwxm" Jul 23 13:07:44.249685 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.249481 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e6fae90e-8282-40c3-9720-4136984b837a-os-release\") pod \"multus-additional-cni-plugins-zwp6q\" (UID: \"e6fae90e-8282-40c3-9720-4136984b837a\") " pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.249685 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.249545 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-lib-modules\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.249685 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.249572 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/04af7f19-0f38-4d43-9b9d-221eb93744c2-socket-dir\") pod \"aws-ebs-csi-driver-node-t27fj\" (UID: \"04af7f19-0f38-4d43-9b9d-221eb93744c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" Jul 23 13:07:44.249685 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.249582 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-ovn-node-metrics-cert\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.249685 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.249619 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b5ef6285-fd8c-4841-bede-d88d25e6ba2b-konnectivity-ca\") pod \"konnectivity-agent-ljjj9\" (UID: \"b5ef6285-fd8c-4841-bede-d88d25e6ba2b\") " pod="kube-system/konnectivity-agent-ljjj9" Jul 23 13:07:44.249685 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.249649 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/04af7f19-0f38-4d43-9b9d-221eb93744c2-sys-fs\") pod \"aws-ebs-csi-driver-node-t27fj\" (UID: \"04af7f19-0f38-4d43-9b9d-221eb93744c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" Jul 23 13:07:44.249685 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.249673 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2lph\" (UniqueName: \"kubernetes.io/projected/04af7f19-0f38-4d43-9b9d-221eb93744c2-kube-api-access-x2lph\") pod \"aws-ebs-csi-driver-node-t27fj\" (UID: \"04af7f19-0f38-4d43-9b9d-221eb93744c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" Jul 23 13:07:44.250035 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.249702 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bdd39476-ed75-4510-a04b-62b9784d80c7-etc-tuned\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.250035 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.249729 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-original-pull-secret\") pod \"global-pull-secret-syncer-vfgv8\" (UID: \"5df5e8b8-2bc3-4788-bf77-4f32a3d52f65\") " pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:07:44.250035 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.249754 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-cnibin\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.250035 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.249782 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnnkh\" (UniqueName: \"kubernetes.io/projected/7af68660-8ebf-43b5-981e-ec1159a7ca1a-kube-api-access-hnnkh\") pod \"node-resolver-xlb6j\" (UID: \"7af68660-8ebf-43b5-981e-ec1159a7ca1a\") " pod="openshift-dns/node-resolver-xlb6j" Jul 23 13:07:44.250035 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.249782 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/04af7f19-0f38-4d43-9b9d-221eb93744c2-sys-fs\") pod \"aws-ebs-csi-driver-node-t27fj\" (UID: \"04af7f19-0f38-4d43-9b9d-221eb93744c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" Jul 23 13:07:44.250035 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.249820 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6fae90e-8282-40c3-9720-4136984b837a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zwp6q\" (UID: \"e6fae90e-8282-40c3-9720-4136984b837a\") " pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.250035 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.249970 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-multus-cni-dir\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.250035 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.249997 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5839c342-f734-4f2e-8093-398fe2d82ed9-multus-daemon-config\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.250466 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250043 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7af68660-8ebf-43b5-981e-ec1159a7ca1a-hosts-file\") pod \"node-resolver-xlb6j\" (UID: \"7af68660-8ebf-43b5-981e-ec1159a7ca1a\") " pod="openshift-dns/node-resolver-xlb6j" Jul 23 13:07:44.250466 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250066 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-var-lib-openvswitch\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.250466 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250084 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5839c342-f734-4f2e-8093-398fe2d82ed9-cni-binary-copy\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.250466 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250100 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-host-var-lib-cni-bin\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.250466 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250141 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-multus-conf-dir\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.250466 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250159 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/04af7f19-0f38-4d43-9b9d-221eb93744c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-t27fj\" (UID: \"04af7f19-0f38-4d43-9b9d-221eb93744c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" Jul 23 13:07:44.250466 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250175 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-run-ovn\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.250466 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250202 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b5ef6285-fd8c-4841-bede-d88d25e6ba2b-agent-certs\") pod \"konnectivity-agent-ljjj9\" (UID: \"b5ef6285-fd8c-4841-bede-d88d25e6ba2b\") " pod="kube-system/konnectivity-agent-ljjj9" Jul 23 13:07:44.250466 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250219 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/04af7f19-0f38-4d43-9b9d-221eb93744c2-device-dir\") pod \"aws-ebs-csi-driver-node-t27fj\" (UID: \"04af7f19-0f38-4d43-9b9d-221eb93744c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" Jul 23 13:07:44.250466 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250263 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-run-systemd\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.250466 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250298 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/04af7f19-0f38-4d43-9b9d-221eb93744c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-t27fj\" (UID: \"04af7f19-0f38-4d43-9b9d-221eb93744c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" Jul 23 13:07:44.250466 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250346 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-ovnkube-script-lib\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.250466 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250378 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/04af7f19-0f38-4d43-9b9d-221eb93744c2-device-dir\") pod \"aws-ebs-csi-driver-node-t27fj\" (UID: \"04af7f19-0f38-4d43-9b9d-221eb93744c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" Jul 23 13:07:44.250466 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250393 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e6fae90e-8282-40c3-9720-4136984b837a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zwp6q\" (UID: \"e6fae90e-8282-40c3-9720-4136984b837a\") " pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.250466 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250424 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/04af7f19-0f38-4d43-9b9d-221eb93744c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t27fj\" (UID: \"04af7f19-0f38-4d43-9b9d-221eb93744c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" Jul 23 13:07:44.250466 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250467 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-etc-sysctl-conf\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.251244 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250474 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b5ef6285-fd8c-4841-bede-d88d25e6ba2b-konnectivity-ca\") pod \"konnectivity-agent-ljjj9\" (UID: \"b5ef6285-fd8c-4841-bede-d88d25e6ba2b\") " pod="kube-system/konnectivity-agent-ljjj9" Jul 23 13:07:44.251244 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250488 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-run\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.251244 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250548 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/04af7f19-0f38-4d43-9b9d-221eb93744c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t27fj\" (UID: \"04af7f19-0f38-4d43-9b9d-221eb93744c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" Jul 23 13:07:44.251244 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250558 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-host-var-lib-kubelet\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.251244 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250587 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-hostroot\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.251244 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250618 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9fbx\" (UniqueName: \"kubernetes.io/projected/5839c342-f734-4f2e-8093-398fe2d82ed9-kube-api-access-q9fbx\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.251244 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250630 2531 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 23 13:07:44.251244 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250646 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6fae90e-8282-40c3-9720-4136984b837a-system-cni-dir\") pod \"multus-additional-cni-plugins-zwp6q\" (UID: \"e6fae90e-8282-40c3-9720-4136984b837a\") " pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.251244 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250675 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pndr\" (UniqueName: \"kubernetes.io/projected/e6fae90e-8282-40c3-9720-4136984b837a-kube-api-access-4pndr\") pod \"multus-additional-cni-plugins-zwp6q\" (UID: \"e6fae90e-8282-40c3-9720-4136984b837a\") " pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.251244 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250710 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/04af7f19-0f38-4d43-9b9d-221eb93744c2-registration-dir\") pod \"aws-ebs-csi-driver-node-t27fj\" (UID: \"04af7f19-0f38-4d43-9b9d-221eb93744c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" Jul 23 13:07:44.251244 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250743 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-etc-sysctl-d\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.251244 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250768 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-etc-systemd\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.251244 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250795 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwlgj\" (UniqueName: \"kubernetes.io/projected/007b5917-9a4d-45b8-8ce2-e1639c8db863-kube-api-access-cwlgj\") pod \"node-ca-f9q95\" (UID: \"007b5917-9a4d-45b8-8ce2-e1639c8db863\") " pod="openshift-image-registry/node-ca-f9q95" Jul 23 13:07:44.251244 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250792 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/04af7f19-0f38-4d43-9b9d-221eb93744c2-registration-dir\") pod \"aws-ebs-csi-driver-node-t27fj\" (UID: \"04af7f19-0f38-4d43-9b9d-221eb93744c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" Jul 23 13:07:44.251244 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250819 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-log-socket\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.251244 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250864 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-host-cni-netd\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.251244 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250891 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7af68660-8ebf-43b5-981e-ec1159a7ca1a-tmp-dir\") pod \"node-resolver-xlb6j\" (UID: \"7af68660-8ebf-43b5-981e-ec1159a7ca1a\") " pod="openshift-dns/node-resolver-xlb6j" Jul 23 13:07:44.251985 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250915 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-host-slash\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.251985 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250943 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-kubelet-config\") pod \"global-pull-secret-syncer-vfgv8\" (UID: \"5df5e8b8-2bc3-4788-bf77-4f32a3d52f65\") " pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:07:44.251985 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250971 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-host-run-netns\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.251985 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.250994 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-etc-kubernetes\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.251985 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251024 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m2mh\" (UniqueName: \"kubernetes.io/projected/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-kube-api-access-7m2mh\") pod \"network-metrics-daemon-cp74b\" (UID: \"e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3\") " pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:07:44.251985 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251052 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dc52fdf9-2a89-4d34-8c45-4ff335a8ca85-iptables-alerter-script\") pod \"iptables-alerter-vbwxm\" (UID: \"dc52fdf9-2a89-4d34-8c45-4ff335a8ca85\") " pod="openshift-network-operator/iptables-alerter-vbwxm" Jul 23 13:07:44.251985 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251077 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dc52fdf9-2a89-4d34-8c45-4ff335a8ca85-host-slash\") pod \"iptables-alerter-vbwxm\" (UID: \"dc52fdf9-2a89-4d34-8c45-4ff335a8ca85\") " pod="openshift-network-operator/iptables-alerter-vbwxm" Jul 23 13:07:44.251985 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251101 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-host-run-netns\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.251985 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251125 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-etc-openvswitch\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.251985 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251151 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-os-release\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.251985 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251187 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-host-run-k8s-cni-cncf-io\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.251985 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251219 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2nht\" (UniqueName: \"kubernetes.io/projected/bdd39476-ed75-4510-a04b-62b9784d80c7-kube-api-access-s2nht\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.251985 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251242 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-sys\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.251985 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251275 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/007b5917-9a4d-45b8-8ce2-e1639c8db863-serviceca\") pod \"node-ca-f9q95\" (UID: \"007b5917-9a4d-45b8-8ce2-e1639c8db863\") " pod="openshift-image-registry/node-ca-f9q95" Jul 23 13:07:44.251985 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251300 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-run-openvswitch\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.251985 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251325 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-host-run-ovn-kubernetes\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.252627 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251352 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-host-var-lib-cni-multus\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.252627 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251377 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtbcw\" (UniqueName: \"kubernetes.io/projected/65c6347a-d5cc-4964-b5dd-b5b4098e0c5e-kube-api-access-jtbcw\") pod \"network-check-target-tc68c\" (UID: \"65c6347a-d5cc-4964-b5dd-b5b4098e0c5e\") " pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:07:44.252627 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251426 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e6fae90e-8282-40c3-9720-4136984b837a-cnibin\") pod \"multus-additional-cni-plugins-zwp6q\" (UID: \"e6fae90e-8282-40c3-9720-4136984b837a\") " pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.252627 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251449 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-etc-kubernetes\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.252627 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251472 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/007b5917-9a4d-45b8-8ce2-e1639c8db863-host\") pod \"node-ca-f9q95\" (UID: \"007b5917-9a4d-45b8-8ce2-e1639c8db863\") " pod="openshift-image-registry/node-ca-f9q95" Jul 23 13:07:44.252627 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251504 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-multus-socket-dir-parent\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.252627 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251545 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-host-run-multus-certs\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.252627 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251590 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e6fae90e-8282-40c3-9720-4136984b837a-cni-binary-copy\") pod \"multus-additional-cni-plugins-zwp6q\" (UID: \"e6fae90e-8282-40c3-9720-4136984b837a\") " pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.252627 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251639 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e6fae90e-8282-40c3-9720-4136984b837a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zwp6q\" (UID: \"e6fae90e-8282-40c3-9720-4136984b837a\") " pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.252627 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251665 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-etc-sysconfig\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.252627 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251681 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-systemd-units\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.252627 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251723 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.252627 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251752 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-dbus\") pod \"global-pull-secret-syncer-vfgv8\" (UID: \"5df5e8b8-2bc3-4788-bf77-4f32a3d52f65\") " pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:07:44.252627 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251777 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-system-cni-dir\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.252627 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251799 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-host\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.252627 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251833 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-host-kubelet\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.253200 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251858 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-host-cni-bin\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.253200 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251884 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-ovnkube-config\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.253200 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251907 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-env-overrides\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.253200 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251932 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5dfc\" (UniqueName: \"kubernetes.io/projected/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-kube-api-access-m5dfc\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.253200 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251961 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdd39476-ed75-4510-a04b-62b9784d80c7-tmp\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.253200 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.251995 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-metrics-certs\") pod \"network-metrics-daemon-cp74b\" (UID: \"e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3\") " pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:07:44.253200 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:44.252108 2531 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:07:44.253200 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:44.252187 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-metrics-certs podName:e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3 nodeName:}" failed. No retries permitted until 2026-07-23 13:07:44.752163126 +0000 UTC m=+1.991439103 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-metrics-certs") pod "network-metrics-daemon-cp74b" (UID: "e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:07:44.254679 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.254582 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b5ef6285-fd8c-4841-bede-d88d25e6ba2b-agent-certs\") pod \"konnectivity-agent-ljjj9\" (UID: \"b5ef6285-fd8c-4841-bede-d88d25e6ba2b\") " pod="kube-system/konnectivity-agent-ljjj9" Jul 23 13:07:44.259249 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.259229 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2lph\" (UniqueName: \"kubernetes.io/projected/04af7f19-0f38-4d43-9b9d-221eb93744c2-kube-api-access-x2lph\") pod \"aws-ebs-csi-driver-node-t27fj\" (UID: \"04af7f19-0f38-4d43-9b9d-221eb93744c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" Jul 23 13:07:44.259733 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.259713 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m2mh\" (UniqueName: \"kubernetes.io/projected/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-kube-api-access-7m2mh\") pod \"network-metrics-daemon-cp74b\" (UID: \"e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3\") " pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:07:44.328285 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:44.328238 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b3cd48ad004afdc21879d1579b5d29b.slice/crio-0049fc3be5d2ececc38ca8d3b3397b08ae9f27cbdafb0c05ad23b42bf794f7c1 WatchSource:0}: Error finding container 0049fc3be5d2ececc38ca8d3b3397b08ae9f27cbdafb0c05ad23b42bf794f7c1: Status 404 returned error can't find the container with id 0049fc3be5d2ececc38ca8d3b3397b08ae9f27cbdafb0c05ad23b42bf794f7c1 Jul 23 13:07:44.334985 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.332887 2531 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 13:07:44.352811 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.352783 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-system-cni-dir\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.352811 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.352812 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-host\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.352950 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.352829 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-host-kubelet\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.352950 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.352845 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-host-cni-bin\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.352950 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.352872 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-ovnkube-config\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.352950 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.352890 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-env-overrides\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.352950 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.352914 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5dfc\" (UniqueName: \"kubernetes.io/projected/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-kube-api-access-m5dfc\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.352950 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.352938 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdd39476-ed75-4510-a04b-62b9784d80c7-tmp\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.352950 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.352914 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-host\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.352950 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.352934 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-system-cni-dir\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.353225 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.352940 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-host-kubelet\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.353225 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.352934 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-host-cni-bin\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.353225 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.352995 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-etc-modprobe-d\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.353225 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353094 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-etc-modprobe-d\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.353225 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353101 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-node-log\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.353225 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353141 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-node-log\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.353225 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353139 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-var-lib-kubelet\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.353225 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353182 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfzht\" (UniqueName: \"kubernetes.io/projected/dc52fdf9-2a89-4d34-8c45-4ff335a8ca85-kube-api-access-pfzht\") pod \"iptables-alerter-vbwxm\" (UID: \"dc52fdf9-2a89-4d34-8c45-4ff335a8ca85\") " pod="openshift-network-operator/iptables-alerter-vbwxm" Jul 23 13:07:44.353225 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353187 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-var-lib-kubelet\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.353225 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353200 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e6fae90e-8282-40c3-9720-4136984b837a-os-release\") pod \"multus-additional-cni-plugins-zwp6q\" (UID: \"e6fae90e-8282-40c3-9720-4136984b837a\") " pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.353225 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353226 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-lib-modules\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.353719 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353242 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-ovn-node-metrics-cert\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.353719 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353260 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bdd39476-ed75-4510-a04b-62b9784d80c7-etc-tuned\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.353719 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353276 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-original-pull-secret\") pod \"global-pull-secret-syncer-vfgv8\" (UID: \"5df5e8b8-2bc3-4788-bf77-4f32a3d52f65\") " pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:07:44.353719 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353291 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-cnibin\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.353719 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353305 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnnkh\" (UniqueName: \"kubernetes.io/projected/7af68660-8ebf-43b5-981e-ec1159a7ca1a-kube-api-access-hnnkh\") pod \"node-resolver-xlb6j\" (UID: \"7af68660-8ebf-43b5-981e-ec1159a7ca1a\") " pod="openshift-dns/node-resolver-xlb6j" Jul 23 13:07:44.353719 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353321 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6fae90e-8282-40c3-9720-4136984b837a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zwp6q\" (UID: \"e6fae90e-8282-40c3-9720-4136984b837a\") " pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.353719 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353336 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-multus-cni-dir\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.353719 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353350 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5839c342-f734-4f2e-8093-398fe2d82ed9-multus-daemon-config\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.353719 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353370 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7af68660-8ebf-43b5-981e-ec1159a7ca1a-hosts-file\") pod \"node-resolver-xlb6j\" (UID: \"7af68660-8ebf-43b5-981e-ec1159a7ca1a\") " pod="openshift-dns/node-resolver-xlb6j" Jul 23 13:07:44.353719 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353390 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-var-lib-openvswitch\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.353719 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353406 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5839c342-f734-4f2e-8093-398fe2d82ed9-cni-binary-copy\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.353719 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353422 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-host-var-lib-cni-bin\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.353719 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353480 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-multus-conf-dir\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.353719 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353483 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-env-overrides\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.353719 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353495 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7af68660-8ebf-43b5-981e-ec1159a7ca1a-hosts-file\") pod \"node-resolver-xlb6j\" (UID: \"7af68660-8ebf-43b5-981e-ec1159a7ca1a\") " pod="openshift-dns/node-resolver-xlb6j" Jul 23 13:07:44.353719 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353504 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-run-ovn\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.353719 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353512 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-ovnkube-config\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.353719 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353516 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e6fae90e-8282-40c3-9720-4136984b837a-os-release\") pod \"multus-additional-cni-plugins-zwp6q\" (UID: \"e6fae90e-8282-40c3-9720-4136984b837a\") " pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.354888 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353531 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-multus-cni-dir\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.354888 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353542 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-run-ovn\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.354888 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353552 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-run-systemd\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.354888 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353581 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-var-lib-openvswitch\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.354888 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:44.353452 2531 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 13:07:44.354888 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353619 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-run-systemd\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.354888 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:44.353656 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-original-pull-secret podName:5df5e8b8-2bc3-4788-bf77-4f32a3d52f65 nodeName:}" failed. No retries permitted until 2026-07-23 13:07:44.853638194 +0000 UTC m=+2.092914150 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-original-pull-secret") pod "global-pull-secret-syncer-vfgv8" (UID: "5df5e8b8-2bc3-4788-bf77-4f32a3d52f65") : object "kube-system"/"original-pull-secret" not registered Jul 23 13:07:44.354888 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353663 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-multus-conf-dir\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.354888 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353580 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-ovnkube-script-lib\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.354888 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353683 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6fae90e-8282-40c3-9720-4136984b837a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zwp6q\" (UID: \"e6fae90e-8282-40c3-9720-4136984b837a\") " pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.354888 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353725 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e6fae90e-8282-40c3-9720-4136984b837a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zwp6q\" (UID: \"e6fae90e-8282-40c3-9720-4136984b837a\") " pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.354888 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353760 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-cnibin\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.354888 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353761 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-etc-sysctl-conf\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.354888 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353792 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-run\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.354888 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353815 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-host-var-lib-kubelet\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.354888 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353838 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-hostroot\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.354888 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353868 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-etc-sysctl-conf\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.355789 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353882 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9fbx\" (UniqueName: \"kubernetes.io/projected/5839c342-f734-4f2e-8093-398fe2d82ed9-kube-api-access-q9fbx\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.355789 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353935 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6fae90e-8282-40c3-9720-4136984b837a-system-cni-dir\") pod \"multus-additional-cni-plugins-zwp6q\" (UID: \"e6fae90e-8282-40c3-9720-4136984b837a\") " pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.355789 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353962 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pndr\" (UniqueName: \"kubernetes.io/projected/e6fae90e-8282-40c3-9720-4136984b837a-kube-api-access-4pndr\") pod \"multus-additional-cni-plugins-zwp6q\" (UID: \"e6fae90e-8282-40c3-9720-4136984b837a\") " pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.355789 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.353971 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-lib-modules\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.355789 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354033 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-host-var-lib-cni-bin\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.355789 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354070 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5839c342-f734-4f2e-8093-398fe2d82ed9-multus-daemon-config\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.355789 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354136 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-host-var-lib-kubelet\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.355789 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354240 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-etc-sysctl-d\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.355789 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354270 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-etc-systemd\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.355789 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354297 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwlgj\" (UniqueName: \"kubernetes.io/projected/007b5917-9a4d-45b8-8ce2-e1639c8db863-kube-api-access-cwlgj\") pod \"node-ca-f9q95\" (UID: \"007b5917-9a4d-45b8-8ce2-e1639c8db863\") " pod="openshift-image-registry/node-ca-f9q95" Jul 23 13:07:44.355789 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354323 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-log-socket\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.355789 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354348 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-host-cni-netd\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.355789 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354355 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5839c342-f734-4f2e-8093-398fe2d82ed9-cni-binary-copy\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.355789 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354371 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7af68660-8ebf-43b5-981e-ec1159a7ca1a-tmp-dir\") pod \"node-resolver-xlb6j\" (UID: \"7af68660-8ebf-43b5-981e-ec1159a7ca1a\") " pod="openshift-dns/node-resolver-xlb6j" Jul 23 13:07:44.355789 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354396 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-host-slash\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.355789 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354423 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-kubelet-config\") pod \"global-pull-secret-syncer-vfgv8\" (UID: \"5df5e8b8-2bc3-4788-bf77-4f32a3d52f65\") " pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:07:44.355789 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354465 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-ovnkube-script-lib\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.355789 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354477 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-run\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.356415 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354469 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-host-run-netns\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.356415 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354511 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-log-socket\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.356415 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354526 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-host-run-netns\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.356415 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354528 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-etc-kubernetes\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.356415 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354552 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-host-cni-netd\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.356415 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354562 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dc52fdf9-2a89-4d34-8c45-4ff335a8ca85-iptables-alerter-script\") pod \"iptables-alerter-vbwxm\" (UID: \"dc52fdf9-2a89-4d34-8c45-4ff335a8ca85\") " pod="openshift-network-operator/iptables-alerter-vbwxm" Jul 23 13:07:44.356415 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354580 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-hostroot\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.356415 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354596 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dc52fdf9-2a89-4d34-8c45-4ff335a8ca85-host-slash\") pod \"iptables-alerter-vbwxm\" (UID: \"dc52fdf9-2a89-4d34-8c45-4ff335a8ca85\") " pod="openshift-network-operator/iptables-alerter-vbwxm" Jul 23 13:07:44.356415 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354622 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-host-run-netns\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.356415 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354628 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6fae90e-8282-40c3-9720-4136984b837a-system-cni-dir\") pod \"multus-additional-cni-plugins-zwp6q\" (UID: \"e6fae90e-8282-40c3-9720-4136984b837a\") " pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.356415 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354664 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-etc-kubernetes\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.356415 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354667 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-host-run-netns\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.356415 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354710 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e6fae90e-8282-40c3-9720-4136984b837a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zwp6q\" (UID: \"e6fae90e-8282-40c3-9720-4136984b837a\") " pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.356415 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354728 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-etc-sysctl-d\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.356415 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354778 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dc52fdf9-2a89-4d34-8c45-4ff335a8ca85-host-slash\") pod \"iptables-alerter-vbwxm\" (UID: \"dc52fdf9-2a89-4d34-8c45-4ff335a8ca85\") " pod="openshift-network-operator/iptables-alerter-vbwxm" Jul 23 13:07:44.356415 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354783 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-etc-systemd\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.356415 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354811 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-etc-openvswitch\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.356415 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354834 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-host-slash\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.356989 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354841 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-os-release\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.356989 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354849 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7af68660-8ebf-43b5-981e-ec1159a7ca1a-tmp-dir\") pod \"node-resolver-xlb6j\" (UID: \"7af68660-8ebf-43b5-981e-ec1159a7ca1a\") " pod="openshift-dns/node-resolver-xlb6j" Jul 23 13:07:44.356989 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354867 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-host-run-k8s-cni-cncf-io\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.356989 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354888 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-etc-openvswitch\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.356989 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354895 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2nht\" (UniqueName: \"kubernetes.io/projected/bdd39476-ed75-4510-a04b-62b9784d80c7-kube-api-access-s2nht\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.356989 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354910 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-kubelet-config\") pod \"global-pull-secret-syncer-vfgv8\" (UID: \"5df5e8b8-2bc3-4788-bf77-4f32a3d52f65\") " pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:07:44.356989 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354922 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-sys\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.356989 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354933 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-os-release\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.356989 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354939 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-host-run-k8s-cni-cncf-io\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.356989 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354952 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/007b5917-9a4d-45b8-8ce2-e1639c8db863-serviceca\") pod \"node-ca-f9q95\" (UID: \"007b5917-9a4d-45b8-8ce2-e1639c8db863\") " pod="openshift-image-registry/node-ca-f9q95" Jul 23 13:07:44.356989 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354977 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-run-openvswitch\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.356989 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354988 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-sys\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.356989 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.354998 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-host-run-ovn-kubernetes\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.356989 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355017 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-host-var-lib-cni-multus\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.356989 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355040 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtbcw\" (UniqueName: \"kubernetes.io/projected/65c6347a-d5cc-4964-b5dd-b5b4098e0c5e-kube-api-access-jtbcw\") pod \"network-check-target-tc68c\" (UID: \"65c6347a-d5cc-4964-b5dd-b5b4098e0c5e\") " pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:07:44.356989 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355064 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-host-run-ovn-kubernetes\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.356989 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355067 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e6fae90e-8282-40c3-9720-4136984b837a-cnibin\") pod \"multus-additional-cni-plugins-zwp6q\" (UID: \"e6fae90e-8282-40c3-9720-4136984b837a\") " pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.356989 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355100 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e6fae90e-8282-40c3-9720-4136984b837a-cnibin\") pod \"multus-additional-cni-plugins-zwp6q\" (UID: \"e6fae90e-8282-40c3-9720-4136984b837a\") " pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.357837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355110 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-etc-kubernetes\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.357837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355136 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/007b5917-9a4d-45b8-8ce2-e1639c8db863-host\") pod \"node-ca-f9q95\" (UID: \"007b5917-9a4d-45b8-8ce2-e1639c8db863\") " pod="openshift-image-registry/node-ca-f9q95" Jul 23 13:07:44.357837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355140 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-host-var-lib-cni-multus\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.357837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355141 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-run-openvswitch\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.357837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355161 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-multus-socket-dir-parent\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.357837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355188 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/007b5917-9a4d-45b8-8ce2-e1639c8db863-host\") pod \"node-ca-f9q95\" (UID: \"007b5917-9a4d-45b8-8ce2-e1639c8db863\") " pod="openshift-image-registry/node-ca-f9q95" Jul 23 13:07:44.357837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355194 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-host-run-multus-certs\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.357837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355203 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dc52fdf9-2a89-4d34-8c45-4ff335a8ca85-iptables-alerter-script\") pod \"iptables-alerter-vbwxm\" (UID: \"dc52fdf9-2a89-4d34-8c45-4ff335a8ca85\") " pod="openshift-network-operator/iptables-alerter-vbwxm" Jul 23 13:07:44.357837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355229 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-host-run-multus-certs\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.357837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355279 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e6fae90e-8282-40c3-9720-4136984b837a-cni-binary-copy\") pod \"multus-additional-cni-plugins-zwp6q\" (UID: \"e6fae90e-8282-40c3-9720-4136984b837a\") " pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.357837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355311 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e6fae90e-8282-40c3-9720-4136984b837a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zwp6q\" (UID: \"e6fae90e-8282-40c3-9720-4136984b837a\") " pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.357837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355377 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-etc-kubernetes\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.357837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355385 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/007b5917-9a4d-45b8-8ce2-e1639c8db863-serviceca\") pod \"node-ca-f9q95\" (UID: \"007b5917-9a4d-45b8-8ce2-e1639c8db863\") " pod="openshift-image-registry/node-ca-f9q95" Jul 23 13:07:44.357837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355385 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-etc-sysconfig\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.357837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355442 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5839c342-f734-4f2e-8093-398fe2d82ed9-multus-socket-dir-parent\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.357837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355422 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-systemd-units\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.357837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355482 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-systemd-units\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.358543 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355487 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.358543 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355482 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bdd39476-ed75-4510-a04b-62b9784d80c7-etc-sysconfig\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.358543 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355514 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-dbus\") pod \"global-pull-secret-syncer-vfgv8\" (UID: \"5df5e8b8-2bc3-4788-bf77-4f32a3d52f65\") " pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:07:44.358543 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355543 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.358543 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355632 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-dbus\") pod \"global-pull-secret-syncer-vfgv8\" (UID: \"5df5e8b8-2bc3-4788-bf77-4f32a3d52f65\") " pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:07:44.358543 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355909 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e6fae90e-8282-40c3-9720-4136984b837a-cni-binary-copy\") pod \"multus-additional-cni-plugins-zwp6q\" (UID: \"e6fae90e-8282-40c3-9720-4136984b837a\") " pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.358543 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.355967 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e6fae90e-8282-40c3-9720-4136984b837a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zwp6q\" (UID: \"e6fae90e-8282-40c3-9720-4136984b837a\") " pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.358543 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.356166 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdd39476-ed75-4510-a04b-62b9784d80c7-tmp\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.358543 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.356200 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bdd39476-ed75-4510-a04b-62b9784d80c7-etc-tuned\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.358543 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.357210 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-ovn-node-metrics-cert\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.363110 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:44.362351 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 13:07:44.363110 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:44.362408 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 13:07:44.363110 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:44.362425 2531 projected.go:196] Error preparing data for projected volume kube-api-access-jtbcw for pod openshift-network-diagnostics/network-check-target-tc68c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:07:44.363110 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:44.362489 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65c6347a-d5cc-4964-b5dd-b5b4098e0c5e-kube-api-access-jtbcw podName:65c6347a-d5cc-4964-b5dd-b5b4098e0c5e nodeName:}" failed. No retries permitted until 2026-07-23 13:07:44.862469432 +0000 UTC m=+2.101745387 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jtbcw" (UniqueName: "kubernetes.io/projected/65c6347a-d5cc-4964-b5dd-b5b4098e0c5e-kube-api-access-jtbcw") pod "network-check-target-tc68c" (UID: "65c6347a-d5cc-4964-b5dd-b5b4098e0c5e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:07:44.364131 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.364103 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5dfc\" (UniqueName: \"kubernetes.io/projected/a0a5f6bc-bdb7-46ae-8cb9-79edd5179817-kube-api-access-m5dfc\") pod \"ovnkube-node-t6m8j\" (UID: \"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.364246 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.364138 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwlgj\" (UniqueName: \"kubernetes.io/projected/007b5917-9a4d-45b8-8ce2-e1639c8db863-kube-api-access-cwlgj\") pod \"node-ca-f9q95\" (UID: \"007b5917-9a4d-45b8-8ce2-e1639c8db863\") " pod="openshift-image-registry/node-ca-f9q95" Jul 23 13:07:44.364316 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.364292 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfzht\" (UniqueName: \"kubernetes.io/projected/dc52fdf9-2a89-4d34-8c45-4ff335a8ca85-kube-api-access-pfzht\") pod \"iptables-alerter-vbwxm\" (UID: \"dc52fdf9-2a89-4d34-8c45-4ff335a8ca85\") " pod="openshift-network-operator/iptables-alerter-vbwxm" Jul 23 13:07:44.364400 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.364376 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pndr\" (UniqueName: \"kubernetes.io/projected/e6fae90e-8282-40c3-9720-4136984b837a-kube-api-access-4pndr\") pod \"multus-additional-cni-plugins-zwp6q\" (UID: \"e6fae90e-8282-40c3-9720-4136984b837a\") " pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.364568 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.364550 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnnkh\" (UniqueName: \"kubernetes.io/projected/7af68660-8ebf-43b5-981e-ec1159a7ca1a-kube-api-access-hnnkh\") pod \"node-resolver-xlb6j\" (UID: \"7af68660-8ebf-43b5-981e-ec1159a7ca1a\") " pod="openshift-dns/node-resolver-xlb6j" Jul 23 13:07:44.365275 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.365259 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2nht\" (UniqueName: \"kubernetes.io/projected/bdd39476-ed75-4510-a04b-62b9784d80c7-kube-api-access-s2nht\") pod \"tuned-jk4mp\" (UID: \"bdd39476-ed75-4510-a04b-62b9784d80c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.365833 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.365818 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9fbx\" (UniqueName: \"kubernetes.io/projected/5839c342-f734-4f2e-8093-398fe2d82ed9-kube-api-access-q9fbx\") pod \"multus-gckpb\" (UID: \"5839c342-f734-4f2e-8093-398fe2d82ed9\") " pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.470239 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.470168 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ljjj9" Jul 23 13:07:44.478195 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:44.478171 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5ef6285_fd8c_4841_bede_d88d25e6ba2b.slice/crio-927b2df98ff2de7910e813919155a162ff62045eba7520d51dbbcce68b7942d7 WatchSource:0}: Error finding container 927b2df98ff2de7910e813919155a162ff62045eba7520d51dbbcce68b7942d7: Status 404 returned error can't find the container with id 927b2df98ff2de7910e813919155a162ff62045eba7520d51dbbcce68b7942d7 Jul 23 13:07:44.488099 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.488079 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" Jul 23 13:07:44.495169 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:44.495149 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04af7f19_0f38_4d43_9b9d_221eb93744c2.slice/crio-be1b79acf03472126bae8e789fbb4c7d50af8dcf9562e62c48b602156cdf0684 WatchSource:0}: Error finding container be1b79acf03472126bae8e789fbb4c7d50af8dcf9562e62c48b602156cdf0684: Status 404 returned error can't find the container with id be1b79acf03472126bae8e789fbb4c7d50af8dcf9562e62c48b602156cdf0684 Jul 23 13:07:44.509952 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.509934 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" Jul 23 13:07:44.517342 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:44.517320 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdd39476_ed75_4510_a04b_62b9784d80c7.slice/crio-8891cd291b607a1b38254f4b42a4bc2f638a8e37a29a401c9d75a87ae4f3109d WatchSource:0}: Error finding container 8891cd291b607a1b38254f4b42a4bc2f638a8e37a29a401c9d75a87ae4f3109d: Status 404 returned error can't find the container with id 8891cd291b607a1b38254f4b42a4bc2f638a8e37a29a401c9d75a87ae4f3109d Jul 23 13:07:44.517812 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.517794 2531 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 13:07:44.522054 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.522038 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xlb6j" Jul 23 13:07:44.529431 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:44.529406 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7af68660_8ebf_43b5_981e_ec1159a7ca1a.slice/crio-9589aba8aee60435980460e08e36094056e92b98f22ce1968642f3a3a2e7ac42 WatchSource:0}: Error finding container 9589aba8aee60435980460e08e36094056e92b98f22ce1968642f3a3a2e7ac42: Status 404 returned error can't find the container with id 9589aba8aee60435980460e08e36094056e92b98f22ce1968642f3a3a2e7ac42 Jul 23 13:07:44.529503 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.529465 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-f9q95" Jul 23 13:07:44.537030 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:44.537009 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod007b5917_9a4d_45b8_8ce2_e1639c8db863.slice/crio-527aaff94d6f321a4d4e8629db476a2e973085a6ea88b32cd3e21d79b73b330e WatchSource:0}: Error finding container 527aaff94d6f321a4d4e8629db476a2e973085a6ea88b32cd3e21d79b73b330e: Status 404 returned error can't find the container with id 527aaff94d6f321a4d4e8629db476a2e973085a6ea88b32cd3e21d79b73b330e Jul 23 13:07:44.550460 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.550438 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vbwxm" Jul 23 13:07:44.558900 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:44.558878 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc52fdf9_2a89_4d34_8c45_4ff335a8ca85.slice/crio-fb9ae504dbcf4f8baea054f25776413b4431c7981ab3a6d6a334abdcd64e66e5 WatchSource:0}: Error finding container fb9ae504dbcf4f8baea054f25776413b4431c7981ab3a6d6a334abdcd64e66e5: Status 404 returned error can't find the container with id fb9ae504dbcf4f8baea054f25776413b4431c7981ab3a6d6a334abdcd64e66e5 Jul 23 13:07:44.571122 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.571104 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:07:44.577241 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.577214 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gckpb" Jul 23 13:07:44.578557 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:44.578536 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0a5f6bc_bdb7_46ae_8cb9_79edd5179817.slice/crio-369b5e65c794faf371375660ce95008c1e35356b1544a1d10a71ab7d4ff5304d WatchSource:0}: Error finding container 369b5e65c794faf371375660ce95008c1e35356b1544a1d10a71ab7d4ff5304d: Status 404 returned error can't find the container with id 369b5e65c794faf371375660ce95008c1e35356b1544a1d10a71ab7d4ff5304d Jul 23 13:07:44.585204 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:44.585181 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5839c342_f734_4f2e_8093_398fe2d82ed9.slice/crio-a93ff620c5c8379dc56ed261adb66a92bd618e94d23377cc13e5f0cdb4e72158 WatchSource:0}: Error finding container a93ff620c5c8379dc56ed261adb66a92bd618e94d23377cc13e5f0cdb4e72158: Status 404 returned error can't find the container with id a93ff620c5c8379dc56ed261adb66a92bd618e94d23377cc13e5f0cdb4e72158 Jul 23 13:07:44.590000 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.589983 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zwp6q" Jul 23 13:07:44.599495 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:07:44.599473 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6fae90e_8282_40c3_9720_4136984b837a.slice/crio-593ecef522b109b5ccb2b44675815d1d57ee0ba74a90fb835370311df4ef10f5 WatchSource:0}: Error finding container 593ecef522b109b5ccb2b44675815d1d57ee0ba74a90fb835370311df4ef10f5: Status 404 returned error can't find the container with id 593ecef522b109b5ccb2b44675815d1d57ee0ba74a90fb835370311df4ef10f5 Jul 23 13:07:44.758075 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.757993 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-metrics-certs\") pod \"network-metrics-daemon-cp74b\" (UID: \"e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3\") " pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:07:44.758215 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:44.758109 2531 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:07:44.758215 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:44.758170 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-metrics-certs podName:e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3 nodeName:}" failed. No retries permitted until 2026-07-23 13:07:45.758154764 +0000 UTC m=+2.997430719 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-metrics-certs") pod "network-metrics-daemon-cp74b" (UID: "e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:07:44.859290 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.859257 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-original-pull-secret\") pod \"global-pull-secret-syncer-vfgv8\" (UID: \"5df5e8b8-2bc3-4788-bf77-4f32a3d52f65\") " pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:07:44.859507 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:44.859445 2531 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 13:07:44.859588 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:44.859526 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-original-pull-secret podName:5df5e8b8-2bc3-4788-bf77-4f32a3d52f65 nodeName:}" failed. No retries permitted until 2026-07-23 13:07:45.859504245 +0000 UTC m=+3.098780215 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-original-pull-secret") pod "global-pull-secret-syncer-vfgv8" (UID: "5df5e8b8-2bc3-4788-bf77-4f32a3d52f65") : object "kube-system"/"original-pull-secret" not registered Jul 23 13:07:44.913397 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.913346 2531 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 13:07:44.960107 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:44.960077 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtbcw\" (UniqueName: \"kubernetes.io/projected/65c6347a-d5cc-4964-b5dd-b5b4098e0c5e-kube-api-access-jtbcw\") pod \"network-check-target-tc68c\" (UID: \"65c6347a-d5cc-4964-b5dd-b5b4098e0c5e\") " pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:07:44.960300 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:44.960232 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 13:07:44.960445 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:44.960352 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 13:07:44.960445 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:44.960368 2531 projected.go:196] Error preparing data for projected volume kube-api-access-jtbcw for pod openshift-network-diagnostics/network-check-target-tc68c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:07:44.960556 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:44.960460 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65c6347a-d5cc-4964-b5dd-b5b4098e0c5e-kube-api-access-jtbcw podName:65c6347a-d5cc-4964-b5dd-b5b4098e0c5e nodeName:}" failed. No retries permitted until 2026-07-23 13:07:45.960437902 +0000 UTC m=+3.199713861 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jtbcw" (UniqueName: "kubernetes.io/projected/65c6347a-d5cc-4964-b5dd-b5b4098e0c5e-kube-api-access-jtbcw") pod "network-check-target-tc68c" (UID: "65c6347a-d5cc-4964-b5dd-b5b4098e0c5e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:07:45.203680 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:45.203530 2531 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 13:02:44 +0000 UTC" deadline="2028-04-15 04:56:35.50070761 +0000 UTC" Jul 23 13:07:45.203680 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:45.203573 2531 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15159h48m50.297139878s" Jul 23 13:07:45.344684 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:45.342819 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:07:45.344684 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:45.343008 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cp74b" podUID="e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3" Jul 23 13:07:45.364805 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:45.364742 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-f9q95" event={"ID":"007b5917-9a4d-45b8-8ce2-e1639c8db863","Type":"ContainerStarted","Data":"527aaff94d6f321a4d4e8629db476a2e973085a6ea88b32cd3e21d79b73b330e"} Jul 23 13:07:45.381413 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:45.381372 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xlb6j" event={"ID":"7af68660-8ebf-43b5-981e-ec1159a7ca1a","Type":"ContainerStarted","Data":"9589aba8aee60435980460e08e36094056e92b98f22ce1968642f3a3a2e7ac42"} Jul 23 13:07:45.415693 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:45.415652 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" event={"ID":"04af7f19-0f38-4d43-9b9d-221eb93744c2","Type":"ContainerStarted","Data":"be1b79acf03472126bae8e789fbb4c7d50af8dcf9562e62c48b602156cdf0684"} Jul 23 13:07:45.426641 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:45.426607 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ljjj9" event={"ID":"b5ef6285-fd8c-4841-bede-d88d25e6ba2b","Type":"ContainerStarted","Data":"927b2df98ff2de7910e813919155a162ff62045eba7520d51dbbcce68b7942d7"} Jul 23 13:07:45.445115 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:45.445066 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-1.ec2.internal" event={"ID":"e2041f17d49a605f250f8febf662052c","Type":"ContainerStarted","Data":"52d1ce75f9110f55f86b61c087b6606aedf85dcbc7cc346229b8a041a5b4f439"} Jul 23 13:07:45.460634 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:45.460548 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-1.ec2.internal" event={"ID":"4b3cd48ad004afdc21879d1579b5d29b","Type":"ContainerStarted","Data":"0049fc3be5d2ececc38ca8d3b3397b08ae9f27cbdafb0c05ad23b42bf794f7c1"} Jul 23 13:07:45.468665 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:45.468626 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwp6q" event={"ID":"e6fae90e-8282-40c3-9720-4136984b837a","Type":"ContainerStarted","Data":"593ecef522b109b5ccb2b44675815d1d57ee0ba74a90fb835370311df4ef10f5"} Jul 23 13:07:45.494490 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:45.494448 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gckpb" event={"ID":"5839c342-f734-4f2e-8093-398fe2d82ed9","Type":"ContainerStarted","Data":"a93ff620c5c8379dc56ed261adb66a92bd618e94d23377cc13e5f0cdb4e72158"} Jul 23 13:07:45.508949 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:45.508917 2531 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 13:07:45.527899 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:45.527859 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vbwxm" event={"ID":"dc52fdf9-2a89-4d34-8c45-4ff335a8ca85","Type":"ContainerStarted","Data":"fb9ae504dbcf4f8baea054f25776413b4431c7981ab3a6d6a334abdcd64e66e5"} Jul 23 13:07:45.546810 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:45.546744 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" event={"ID":"bdd39476-ed75-4510-a04b-62b9784d80c7","Type":"ContainerStarted","Data":"8891cd291b607a1b38254f4b42a4bc2f638a8e37a29a401c9d75a87ae4f3109d"} Jul 23 13:07:45.576338 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:45.576215 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" event={"ID":"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817","Type":"ContainerStarted","Data":"369b5e65c794faf371375660ce95008c1e35356b1544a1d10a71ab7d4ff5304d"} Jul 23 13:07:45.768179 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:45.768091 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-metrics-certs\") pod \"network-metrics-daemon-cp74b\" (UID: \"e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3\") " pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:07:45.768344 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:45.768280 2531 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:07:45.768401 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:45.768351 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-metrics-certs podName:e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3 nodeName:}" failed. No retries permitted until 2026-07-23 13:07:47.768329679 +0000 UTC m=+5.007605640 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-metrics-certs") pod "network-metrics-daemon-cp74b" (UID: "e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:07:45.848716 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:45.848682 2531 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 13:07:45.871453 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:45.871401 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-original-pull-secret\") pod \"global-pull-secret-syncer-vfgv8\" (UID: \"5df5e8b8-2bc3-4788-bf77-4f32a3d52f65\") " pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:07:45.871643 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:45.871624 2531 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 13:07:45.871705 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:45.871693 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-original-pull-secret podName:5df5e8b8-2bc3-4788-bf77-4f32a3d52f65 nodeName:}" failed. No retries permitted until 2026-07-23 13:07:47.87167349 +0000 UTC m=+5.110949473 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-original-pull-secret") pod "global-pull-secret-syncer-vfgv8" (UID: "5df5e8b8-2bc3-4788-bf77-4f32a3d52f65") : object "kube-system"/"original-pull-secret" not registered Jul 23 13:07:45.972734 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:45.972695 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtbcw\" (UniqueName: \"kubernetes.io/projected/65c6347a-d5cc-4964-b5dd-b5b4098e0c5e-kube-api-access-jtbcw\") pod \"network-check-target-tc68c\" (UID: \"65c6347a-d5cc-4964-b5dd-b5b4098e0c5e\") " pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:07:45.972938 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:45.972926 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 13:07:45.973012 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:45.972951 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 13:07:45.973012 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:45.972965 2531 projected.go:196] Error preparing data for projected volume kube-api-access-jtbcw for pod openshift-network-diagnostics/network-check-target-tc68c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:07:45.973122 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:45.973026 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65c6347a-d5cc-4964-b5dd-b5b4098e0c5e-kube-api-access-jtbcw podName:65c6347a-d5cc-4964-b5dd-b5b4098e0c5e nodeName:}" failed. No retries permitted until 2026-07-23 13:07:47.973007824 +0000 UTC m=+5.212283782 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jtbcw" (UniqueName: "kubernetes.io/projected/65c6347a-d5cc-4964-b5dd-b5b4098e0c5e-kube-api-access-jtbcw") pod "network-check-target-tc68c" (UID: "65c6347a-d5cc-4964-b5dd-b5b4098e0c5e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:07:46.204247 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:46.204095 2531 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 13:02:44 +0000 UTC" deadline="2028-04-27 05:22:57.333880928 +0000 UTC" Jul 23 13:07:46.204247 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:46.204197 2531 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15448h15m11.129689344s" Jul 23 13:07:46.326774 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:46.326634 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:07:46.326774 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:46.326634 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:07:46.327018 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:46.326782 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc68c" podUID="65c6347a-d5cc-4964-b5dd-b5b4098e0c5e" Jul 23 13:07:46.327018 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:46.326886 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfgv8" podUID="5df5e8b8-2bc3-4788-bf77-4f32a3d52f65" Jul 23 13:07:47.326029 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:47.325993 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:07:47.326536 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:47.326153 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cp74b" podUID="e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3" Jul 23 13:07:47.788409 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:47.788290 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-metrics-certs\") pod \"network-metrics-daemon-cp74b\" (UID: \"e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3\") " pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:07:47.788585 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:47.788462 2531 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:07:47.788585 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:47.788533 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-metrics-certs podName:e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3 nodeName:}" failed. No retries permitted until 2026-07-23 13:07:51.78851083 +0000 UTC m=+9.027786788 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-metrics-certs") pod "network-metrics-daemon-cp74b" (UID: "e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:07:47.889176 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:47.889133 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-original-pull-secret\") pod \"global-pull-secret-syncer-vfgv8\" (UID: \"5df5e8b8-2bc3-4788-bf77-4f32a3d52f65\") " pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:07:47.889350 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:47.889307 2531 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 13:07:47.889455 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:47.889379 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-original-pull-secret podName:5df5e8b8-2bc3-4788-bf77-4f32a3d52f65 nodeName:}" failed. No retries permitted until 2026-07-23 13:07:51.889357609 +0000 UTC m=+9.128633587 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-original-pull-secret") pod "global-pull-secret-syncer-vfgv8" (UID: "5df5e8b8-2bc3-4788-bf77-4f32a3d52f65") : object "kube-system"/"original-pull-secret" not registered Jul 23 13:07:47.990494 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:47.990452 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtbcw\" (UniqueName: \"kubernetes.io/projected/65c6347a-d5cc-4964-b5dd-b5b4098e0c5e-kube-api-access-jtbcw\") pod \"network-check-target-tc68c\" (UID: \"65c6347a-d5cc-4964-b5dd-b5b4098e0c5e\") " pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:07:47.990682 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:47.990645 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 13:07:47.990682 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:47.990667 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 13:07:47.990682 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:47.990679 2531 projected.go:196] Error preparing data for projected volume kube-api-access-jtbcw for pod openshift-network-diagnostics/network-check-target-tc68c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:07:47.990857 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:47.990741 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65c6347a-d5cc-4964-b5dd-b5b4098e0c5e-kube-api-access-jtbcw podName:65c6347a-d5cc-4964-b5dd-b5b4098e0c5e nodeName:}" failed. No retries permitted until 2026-07-23 13:07:51.990721737 +0000 UTC m=+9.229997698 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jtbcw" (UniqueName: "kubernetes.io/projected/65c6347a-d5cc-4964-b5dd-b5b4098e0c5e-kube-api-access-jtbcw") pod "network-check-target-tc68c" (UID: "65c6347a-d5cc-4964-b5dd-b5b4098e0c5e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:07:48.326683 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:48.325941 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:07:48.326683 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:48.326085 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc68c" podUID="65c6347a-d5cc-4964-b5dd-b5b4098e0c5e" Jul 23 13:07:48.326683 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:48.326511 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:07:48.326683 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:48.326616 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfgv8" podUID="5df5e8b8-2bc3-4788-bf77-4f32a3d52f65" Jul 23 13:07:49.330865 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:49.330832 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:07:49.331363 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:49.330987 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cp74b" podUID="e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3" Jul 23 13:07:50.327642 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:50.326873 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:07:50.327642 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:50.327016 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfgv8" podUID="5df5e8b8-2bc3-4788-bf77-4f32a3d52f65" Jul 23 13:07:50.327642 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:50.327477 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:07:50.327642 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:50.327588 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc68c" podUID="65c6347a-d5cc-4964-b5dd-b5b4098e0c5e" Jul 23 13:07:51.327238 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:51.326974 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:07:51.327699 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:51.327362 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cp74b" podUID="e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3" Jul 23 13:07:51.828385 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:51.827764 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-metrics-certs\") pod \"network-metrics-daemon-cp74b\" (UID: \"e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3\") " pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:07:51.828385 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:51.827953 2531 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:07:51.828385 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:51.828030 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-metrics-certs podName:e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3 nodeName:}" failed. No retries permitted until 2026-07-23 13:07:59.828009411 +0000 UTC m=+17.067285368 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-metrics-certs") pod "network-metrics-daemon-cp74b" (UID: "e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:07:51.929523 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:51.929003 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-original-pull-secret\") pod \"global-pull-secret-syncer-vfgv8\" (UID: \"5df5e8b8-2bc3-4788-bf77-4f32a3d52f65\") " pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:07:51.929523 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:51.929142 2531 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 13:07:51.929523 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:51.929200 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-original-pull-secret podName:5df5e8b8-2bc3-4788-bf77-4f32a3d52f65 nodeName:}" failed. No retries permitted until 2026-07-23 13:07:59.929184813 +0000 UTC m=+17.168460769 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-original-pull-secret") pod "global-pull-secret-syncer-vfgv8" (UID: "5df5e8b8-2bc3-4788-bf77-4f32a3d52f65") : object "kube-system"/"original-pull-secret" not registered Jul 23 13:07:52.030065 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:52.030013 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtbcw\" (UniqueName: \"kubernetes.io/projected/65c6347a-d5cc-4964-b5dd-b5b4098e0c5e-kube-api-access-jtbcw\") pod \"network-check-target-tc68c\" (UID: \"65c6347a-d5cc-4964-b5dd-b5b4098e0c5e\") " pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:07:52.030254 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:52.030241 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 13:07:52.030325 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:52.030269 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 13:07:52.030325 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:52.030285 2531 projected.go:196] Error preparing data for projected volume kube-api-access-jtbcw for pod openshift-network-diagnostics/network-check-target-tc68c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:07:52.030474 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:52.030350 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65c6347a-d5cc-4964-b5dd-b5b4098e0c5e-kube-api-access-jtbcw podName:65c6347a-d5cc-4964-b5dd-b5b4098e0c5e nodeName:}" failed. No retries permitted until 2026-07-23 13:08:00.03033186 +0000 UTC m=+17.269607841 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jtbcw" (UniqueName: "kubernetes.io/projected/65c6347a-d5cc-4964-b5dd-b5b4098e0c5e-kube-api-access-jtbcw") pod "network-check-target-tc68c" (UID: "65c6347a-d5cc-4964-b5dd-b5b4098e0c5e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:07:52.326385 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:52.326288 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:07:52.326559 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:52.326451 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc68c" podUID="65c6347a-d5cc-4964-b5dd-b5b4098e0c5e" Jul 23 13:07:52.326813 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:52.326790 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:07:52.326928 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:52.326906 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfgv8" podUID="5df5e8b8-2bc3-4788-bf77-4f32a3d52f65" Jul 23 13:07:53.328607 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:53.327456 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:07:53.328607 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:53.327587 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cp74b" podUID="e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3" Jul 23 13:07:54.326483 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:54.326451 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:07:54.326674 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:54.326464 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:07:54.327062 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:54.326747 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc68c" podUID="65c6347a-d5cc-4964-b5dd-b5b4098e0c5e" Jul 23 13:07:54.327062 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:54.326787 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfgv8" podUID="5df5e8b8-2bc3-4788-bf77-4f32a3d52f65" Jul 23 13:07:55.326983 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:55.326939 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:07:55.327456 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:55.327079 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cp74b" podUID="e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3" Jul 23 13:07:56.326408 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:56.326375 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:07:56.326593 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:56.326488 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfgv8" podUID="5df5e8b8-2bc3-4788-bf77-4f32a3d52f65" Jul 23 13:07:56.326593 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:56.326541 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:07:56.326706 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:56.326625 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc68c" podUID="65c6347a-d5cc-4964-b5dd-b5b4098e0c5e" Jul 23 13:07:57.329487 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:57.329464 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:07:57.329823 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:57.329603 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cp74b" podUID="e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3" Jul 23 13:07:58.326809 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:58.326769 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:07:58.327010 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:58.326817 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:07:58.327010 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:58.326919 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfgv8" podUID="5df5e8b8-2bc3-4788-bf77-4f32a3d52f65" Jul 23 13:07:58.327126 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:58.327019 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc68c" podUID="65c6347a-d5cc-4964-b5dd-b5b4098e0c5e" Jul 23 13:07:59.326248 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:59.326215 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:07:59.326688 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:59.326343 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cp74b" podUID="e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3" Jul 23 13:07:59.891306 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:59.891268 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-metrics-certs\") pod \"network-metrics-daemon-cp74b\" (UID: \"e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3\") " pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:07:59.891551 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:59.891424 2531 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:07:59.891551 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:59.891494 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-metrics-certs podName:e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3 nodeName:}" failed. No retries permitted until 2026-07-23 13:08:15.891473797 +0000 UTC m=+33.130749755 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-metrics-certs") pod "network-metrics-daemon-cp74b" (UID: "e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:07:59.991587 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:07:59.991552 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-original-pull-secret\") pod \"global-pull-secret-syncer-vfgv8\" (UID: \"5df5e8b8-2bc3-4788-bf77-4f32a3d52f65\") " pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:07:59.991728 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:59.991708 2531 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 13:07:59.991804 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:07:59.991790 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-original-pull-secret podName:5df5e8b8-2bc3-4788-bf77-4f32a3d52f65 nodeName:}" failed. No retries permitted until 2026-07-23 13:08:15.991766958 +0000 UTC m=+33.231042919 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-original-pull-secret") pod "global-pull-secret-syncer-vfgv8" (UID: "5df5e8b8-2bc3-4788-bf77-4f32a3d52f65") : object "kube-system"/"original-pull-secret" not registered Jul 23 13:08:00.092387 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:00.092330 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtbcw\" (UniqueName: \"kubernetes.io/projected/65c6347a-d5cc-4964-b5dd-b5b4098e0c5e-kube-api-access-jtbcw\") pod \"network-check-target-tc68c\" (UID: \"65c6347a-d5cc-4964-b5dd-b5b4098e0c5e\") " pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:08:00.092571 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:00.092504 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 13:08:00.092571 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:00.092526 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 13:08:00.092571 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:00.092540 2531 projected.go:196] Error preparing data for projected volume kube-api-access-jtbcw for pod openshift-network-diagnostics/network-check-target-tc68c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:08:00.092711 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:00.092592 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65c6347a-d5cc-4964-b5dd-b5b4098e0c5e-kube-api-access-jtbcw podName:65c6347a-d5cc-4964-b5dd-b5b4098e0c5e nodeName:}" failed. No retries permitted until 2026-07-23 13:08:16.092577512 +0000 UTC m=+33.331853465 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jtbcw" (UniqueName: "kubernetes.io/projected/65c6347a-d5cc-4964-b5dd-b5b4098e0c5e-kube-api-access-jtbcw") pod "network-check-target-tc68c" (UID: "65c6347a-d5cc-4964-b5dd-b5b4098e0c5e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:08:00.325945 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:00.325870 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:08:00.325945 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:00.325898 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:08:00.326143 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:00.325986 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfgv8" podUID="5df5e8b8-2bc3-4788-bf77-4f32a3d52f65" Jul 23 13:08:00.326143 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:00.326099 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc68c" podUID="65c6347a-d5cc-4964-b5dd-b5b4098e0c5e" Jul 23 13:08:01.326156 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:01.326125 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:08:01.326671 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:01.326277 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cp74b" podUID="e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3" Jul 23 13:08:02.326469 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:02.326430 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:08:02.326469 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:02.326460 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:08:02.326957 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:02.326623 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfgv8" podUID="5df5e8b8-2bc3-4788-bf77-4f32a3d52f65" Jul 23 13:08:02.327042 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:02.327014 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc68c" podUID="65c6347a-d5cc-4964-b5dd-b5b4098e0c5e" Jul 23 13:08:03.327396 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:03.327340 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:08:03.327819 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:03.327512 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cp74b" podUID="e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3" Jul 23 13:08:04.326794 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:04.326762 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:08:04.326794 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:04.326802 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:08:04.327038 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:04.326892 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc68c" podUID="65c6347a-d5cc-4964-b5dd-b5b4098e0c5e" Jul 23 13:08:04.327101 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:04.327038 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfgv8" podUID="5df5e8b8-2bc3-4788-bf77-4f32a3d52f65" Jul 23 13:08:05.326847 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:05.326808 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:08:05.327445 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:05.326927 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cp74b" podUID="e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3" Jul 23 13:08:06.326230 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.326040 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:08:06.326394 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:06.326332 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc68c" podUID="65c6347a-d5cc-4964-b5dd-b5b4098e0c5e" Jul 23 13:08:06.326394 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.326069 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:08:06.326493 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:06.326482 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfgv8" podUID="5df5e8b8-2bc3-4788-bf77-4f32a3d52f65" Jul 23 13:08:06.629114 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.628936 2531 generic.go:356] "Generic (PLEG): container finished" podID="e6fae90e-8282-40c3-9720-4136984b837a" containerID="cc99eb82a62ae8bacc4cdb91854da4af3fe18c638b60af908fbbe395403e94ae" exitCode=0 Jul 23 13:08:06.629919 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.629023 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwp6q" event={"ID":"e6fae90e-8282-40c3-9720-4136984b837a","Type":"ContainerDied","Data":"cc99eb82a62ae8bacc4cdb91854da4af3fe18c638b60af908fbbe395403e94ae"} Jul 23 13:08:06.630875 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.630852 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gckpb" event={"ID":"5839c342-f734-4f2e-8093-398fe2d82ed9","Type":"ContainerStarted","Data":"e99381f22485f43a97a619eab46bb03304f69118f397d4aa321bd1b550298bec"} Jul 23 13:08:06.632295 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.632259 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" event={"ID":"bdd39476-ed75-4510-a04b-62b9784d80c7","Type":"ContainerStarted","Data":"4236f181a3fe47181e8b211aaa8b9f8bc3aed3ca07930fd57a162a298e723462"} Jul 23 13:08:06.637967 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.637947 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6m8j_a0a5f6bc-bdb7-46ae-8cb9-79edd5179817/ovn-acl-logging/0.log" Jul 23 13:08:06.638300 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.638281 2531 generic.go:356] "Generic (PLEG): container finished" podID="a0a5f6bc-bdb7-46ae-8cb9-79edd5179817" containerID="3cbf4530db72f260c44ff897f009656f0f865ad04016b6dec7893091a33c3f1d" exitCode=1 Jul 23 13:08:06.638389 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.638303 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" event={"ID":"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817","Type":"ContainerStarted","Data":"4a4ccd7f2ab47d54d8b8a9d31bb2bf7adc9886a151effe6a990dd256b3ed1981"} Jul 23 13:08:06.638433 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.638416 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" event={"ID":"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817","Type":"ContainerStarted","Data":"5a66a7372cf79d84b28eb9881dfd1ea1c6767764fe64b8e950b31fabd79bf0ba"} Jul 23 13:08:06.638474 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.638465 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" event={"ID":"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817","Type":"ContainerStarted","Data":"bdd96a95e933e79c4140641f58ed067a66839bf74d2f8612734769a04bde7021"} Jul 23 13:08:06.638527 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.638513 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" event={"ID":"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817","Type":"ContainerStarted","Data":"64ad5550224d6af4e61823f459539a454cd684c6f718a10567bb56159df03696"} Jul 23 13:08:06.638578 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.638565 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" event={"ID":"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817","Type":"ContainerDied","Data":"3cbf4530db72f260c44ff897f009656f0f865ad04016b6dec7893091a33c3f1d"} Jul 23 13:08:06.638630 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.638618 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" event={"ID":"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817","Type":"ContainerStarted","Data":"034fa92f42f29c34db0835c564f4637b06403ab97906d7e17754f583fd5ac63d"} Jul 23 13:08:06.639709 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.639686 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-f9q95" event={"ID":"007b5917-9a4d-45b8-8ce2-e1639c8db863","Type":"ContainerStarted","Data":"89c5282017192d8f0c3a1c19ade86d457071d9d287195a9b0d03cdc489961d2b"} Jul 23 13:08:06.640923 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.640900 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xlb6j" event={"ID":"7af68660-8ebf-43b5-981e-ec1159a7ca1a","Type":"ContainerStarted","Data":"ffd1040a2fe3090b0b973be49dd374de162be45568dd9182b8b29cca994b4484"} Jul 23 13:08:06.642209 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.642193 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" event={"ID":"04af7f19-0f38-4d43-9b9d-221eb93744c2","Type":"ContainerStarted","Data":"795fe3d0e81b9b372c2367adcacf9c64309566877dffc1d436c4e5026e587ef2"} Jul 23 13:08:06.643541 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.643511 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ljjj9" event={"ID":"b5ef6285-fd8c-4841-bede-d88d25e6ba2b","Type":"ContainerStarted","Data":"0da536727756a9acb3b05ec6d8daa40533f494e611d621fff87fa17ccd0e16e8"} Jul 23 13:08:06.644812 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.644792 2531 generic.go:356] "Generic (PLEG): container finished" podID="e2041f17d49a605f250f8febf662052c" containerID="96a3bc23f50ccde7e071cdf7d7d23fe7bb81dd51c4a8d5799fbb2000764fa9af" exitCode=0 Jul 23 13:08:06.644885 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.644865 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-1.ec2.internal" event={"ID":"e2041f17d49a605f250f8febf662052c","Type":"ContainerDied","Data":"96a3bc23f50ccde7e071cdf7d7d23fe7bb81dd51c4a8d5799fbb2000764fa9af"} Jul 23 13:08:06.646141 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.646121 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-1.ec2.internal" event={"ID":"4b3cd48ad004afdc21879d1579b5d29b","Type":"ContainerStarted","Data":"878f792aaa245a1cd8c709329f2b339231d2ef4ac42be51505927ffcf9d60464"} Jul 23 13:08:06.664770 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.664709 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-f9q95" podStartSLOduration=1.503844356 podStartE2EDuration="22.664691099s" totalImagesPullingTime="21.160846743s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:07:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:08:06.664578873 +0000 UTC m=+23.903854852" watchObservedRunningTime="2026-07-23 13:08:06.664691099 +0000 UTC m=+23.903967064" Jul 23 13:08:06.693588 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.693546 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ljjj9" podStartSLOduration=2.474487022 podStartE2EDuration="23.693524665s" totalImagesPullingTime="21.219037643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:07:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:08:06.6930774 +0000 UTC m=+23.932353377" watchObservedRunningTime="2026-07-23 13:08:06.693524665 +0000 UTC m=+23.932800642" Jul 23 13:08:06.709537 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.709497 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jk4mp" podStartSLOduration=1.527958889 podStartE2EDuration="22.709484487s" totalImagesPullingTime="21.181525598s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:07:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:08:06.709293191 +0000 UTC m=+23.948569168" watchObservedRunningTime="2026-07-23 13:08:06.709484487 +0000 UTC m=+23.948760459" Jul 23 13:08:06.734974 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.734935 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gckpb" podStartSLOduration=1.589963523 podStartE2EDuration="22.734921453s" totalImagesPullingTime="21.14495793s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:07:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:08:06.734700819 +0000 UTC m=+23.973976807" watchObservedRunningTime="2026-07-23 13:08:06.734921453 +0000 UTC m=+23.974197429" Jul 23 13:08:06.749204 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:06.749167 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-1.ec2.internal" podStartSLOduration=22.749152801 podStartE2EDuration="22.749152801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:07:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:08:06.748716584 +0000 UTC m=+23.987992561" watchObservedRunningTime="2026-07-23 13:08:06.749152801 +0000 UTC m=+23.988428774" Jul 23 13:08:07.326678 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:07.326607 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:08:07.326824 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:07.326731 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cp74b" podUID="e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3" Jul 23 13:08:07.650684 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:07.650651 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-1.ec2.internal" event={"ID":"e2041f17d49a605f250f8febf662052c","Type":"ContainerStarted","Data":"8fc35283a834bb8d5e52b0c30d9973b4a1ba55070b19edd2af5f6385af99048e"} Jul 23 13:08:07.652884 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:07.652852 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vbwxm" event={"ID":"dc52fdf9-2a89-4d34-8c45-4ff335a8ca85","Type":"ContainerStarted","Data":"e9ed18314c9278d743b521b0654c4718474a08162ec14ab9faef23ea4c1874b1"} Jul 23 13:08:07.669277 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:07.669234 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-1.ec2.internal" podStartSLOduration=23.669204802 podStartE2EDuration="23.669204802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:07:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:08:07.669180787 +0000 UTC m=+24.908456761" watchObservedRunningTime="2026-07-23 13:08:07.669204802 +0000 UTC m=+24.908480777" Jul 23 13:08:07.669752 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:07.669623 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xlb6j" podStartSLOduration=2.501426078 podStartE2EDuration="23.669612439s" totalImagesPullingTime="21.168186361s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:07:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:08:06.763103273 +0000 UTC m=+24.002379254" watchObservedRunningTime="2026-07-23 13:08:07.669612439 +0000 UTC m=+24.908888452" Jul 23 13:08:07.684174 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:07.684134 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vbwxm" podStartSLOduration=2.545544805 podStartE2EDuration="23.684122972s" totalImagesPullingTime="21.138578167s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:07:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:08:07.683902353 +0000 UTC m=+24.923178330" watchObservedRunningTime="2026-07-23 13:08:07.684122972 +0000 UTC m=+24.923398947" Jul 23 13:08:07.715768 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:07.715746 2531 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 23 13:08:08.220494 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:08.220388 2531 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T13:08:07.715762471Z","UUID":"86f3aa6a-b693-4f9a-9fc6-d5290e35ddf4","Handler":null,"Name":"","Endpoint":""} Jul 23 13:08:08.221834 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:08.221814 2531 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 23 13:08:08.221982 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:08.221853 2531 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 23 13:08:08.326502 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:08.326472 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:08:08.326665 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:08.326595 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc68c" podUID="65c6347a-d5cc-4964-b5dd-b5b4098e0c5e" Jul 23 13:08:08.326665 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:08.326650 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:08:08.326790 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:08.326766 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfgv8" podUID="5df5e8b8-2bc3-4788-bf77-4f32a3d52f65" Jul 23 13:08:08.658345 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:08.658303 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" event={"ID":"04af7f19-0f38-4d43-9b9d-221eb93744c2","Type":"ContainerStarted","Data":"c0cf38f4567fdd6924557a0b929e972d54e902864bcdb8f0ca030b691675bf49"} Jul 23 13:08:08.708572 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:08.708501 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ljjj9" Jul 23 13:08:08.709193 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:08.709172 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ljjj9" Jul 23 13:08:09.326824 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:09.326789 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:08:09.326981 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:09.326944 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cp74b" podUID="e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3" Jul 23 13:08:09.664390 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:09.664342 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6m8j_a0a5f6bc-bdb7-46ae-8cb9-79edd5179817/ovn-acl-logging/0.log" Jul 23 13:08:09.664853 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:09.664826 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" event={"ID":"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817","Type":"ContainerStarted","Data":"34ef349890d17fbc07644f5b4d65ee6d6527e49cc2ec22a90fe95ae4ec7263f2"} Jul 23 13:08:09.667254 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:09.667228 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" event={"ID":"04af7f19-0f38-4d43-9b9d-221eb93744c2","Type":"ContainerStarted","Data":"29d0616b71fea3c5ec17678b36de3efe668c7ef7b5602befcc71fa6a0fa91fb5"} Jul 23 13:08:09.667488 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:09.667464 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ljjj9" Jul 23 13:08:09.667945 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:09.667927 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ljjj9" Jul 23 13:08:09.689421 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:09.689376 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t27fj" podStartSLOduration=2.856412783 podStartE2EDuration="26.689340879s" totalImagesPullingTime="23.832928096s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:07:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:08:09.688671098 +0000 UTC m=+26.927947079" watchObservedRunningTime="2026-07-23 13:08:09.689340879 +0000 UTC m=+26.928616857" Jul 23 13:08:10.326668 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:10.326626 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:08:10.326668 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:10.326654 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:08:10.326905 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:10.326747 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc68c" podUID="65c6347a-d5cc-4964-b5dd-b5b4098e0c5e" Jul 23 13:08:10.326905 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:10.326879 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfgv8" podUID="5df5e8b8-2bc3-4788-bf77-4f32a3d52f65" Jul 23 13:08:11.326026 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:11.325990 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:08:11.326445 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:11.326152 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cp74b" podUID="e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3" Jul 23 13:08:12.326274 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:12.325857 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:08:12.326274 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:12.325924 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:08:12.326274 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:12.326196 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc68c" podUID="65c6347a-d5cc-4964-b5dd-b5b4098e0c5e" Jul 23 13:08:12.326274 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:12.326263 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfgv8" podUID="5df5e8b8-2bc3-4788-bf77-4f32a3d52f65" Jul 23 13:08:12.678249 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:12.678215 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6m8j_a0a5f6bc-bdb7-46ae-8cb9-79edd5179817/ovn-acl-logging/0.log" Jul 23 13:08:12.678676 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:12.678637 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" event={"ID":"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817","Type":"ContainerStarted","Data":"7df14057f688c0e26c479d1b0d74070b3c35eaad34669f0a37652b638cf1d692"} Jul 23 13:08:12.679133 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:12.678938 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:08:12.679133 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:12.679021 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:08:12.679133 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:12.679071 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:08:12.679133 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:12.679080 2531 scope.go:117] "RemoveContainer" containerID="3cbf4530db72f260c44ff897f009656f0f865ad04016b6dec7893091a33c3f1d" Jul 23 13:08:12.680674 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:12.680654 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwp6q" event={"ID":"e6fae90e-8282-40c3-9720-4136984b837a","Type":"ContainerStarted","Data":"601d4909dc9f0404441b7b2b8d9e77293f7bc9a7b640eb1154657cd3a8f199fe"} Jul 23 13:08:12.696256 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:12.696230 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:08:12.700308 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:12.700287 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:08:13.326938 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:13.326908 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:08:13.327596 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:13.326997 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cp74b" podUID="e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3" Jul 23 13:08:13.686456 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:13.686428 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6m8j_a0a5f6bc-bdb7-46ae-8cb9-79edd5179817/ovn-acl-logging/0.log" Jul 23 13:08:13.686791 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:13.686768 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" event={"ID":"a0a5f6bc-bdb7-46ae-8cb9-79edd5179817","Type":"ContainerStarted","Data":"0db90f49f821f384e355ec1e32277e203731448bae80599e89a522478755d1f3"} Jul 23 13:08:13.688489 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:13.688470 2531 generic.go:356] "Generic (PLEG): container finished" podID="e6fae90e-8282-40c3-9720-4136984b837a" containerID="601d4909dc9f0404441b7b2b8d9e77293f7bc9a7b640eb1154657cd3a8f199fe" exitCode=0 Jul 23 13:08:13.688576 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:13.688499 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwp6q" event={"ID":"e6fae90e-8282-40c3-9720-4136984b837a","Type":"ContainerDied","Data":"601d4909dc9f0404441b7b2b8d9e77293f7bc9a7b640eb1154657cd3a8f199fe"} Jul 23 13:08:13.715591 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:13.715550 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" podStartSLOduration=8.487554012 podStartE2EDuration="29.71553659s" totalImagesPullingTime="21.227982578s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:07:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:08:13.714096473 +0000 UTC m=+30.953372450" watchObservedRunningTime="2026-07-23 13:08:13.71553659 +0000 UTC m=+30.954812566" Jul 23 13:08:14.326373 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:14.326327 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:08:14.326540 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:14.326328 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:08:14.326540 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:14.326476 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc68c" podUID="65c6347a-d5cc-4964-b5dd-b5b4098e0c5e" Jul 23 13:08:14.326647 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:14.326588 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfgv8" podUID="5df5e8b8-2bc3-4788-bf77-4f32a3d52f65" Jul 23 13:08:15.327108 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:15.326930 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:08:15.327591 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:15.327211 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cp74b" podUID="e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3" Jul 23 13:08:15.694394 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:15.694298 2531 generic.go:356] "Generic (PLEG): container finished" podID="e6fae90e-8282-40c3-9720-4136984b837a" containerID="739028ea287f0199d8d961b7607abe0ce3a6604e507952357296bf43fbea45ee" exitCode=0 Jul 23 13:08:15.694394 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:15.694343 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwp6q" event={"ID":"e6fae90e-8282-40c3-9720-4136984b837a","Type":"ContainerDied","Data":"739028ea287f0199d8d961b7607abe0ce3a6604e507952357296bf43fbea45ee"} Jul 23 13:08:15.904009 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:15.903972 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-metrics-certs\") pod \"network-metrics-daemon-cp74b\" (UID: \"e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3\") " pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:08:15.904159 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:15.904079 2531 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:08:15.904159 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:15.904130 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-metrics-certs podName:e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3 nodeName:}" failed. No retries permitted until 2026-07-23 13:08:47.904118065 +0000 UTC m=+65.143394019 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-metrics-certs") pod "network-metrics-daemon-cp74b" (UID: "e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:08:16.005147 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:16.005065 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-original-pull-secret\") pod \"global-pull-secret-syncer-vfgv8\" (UID: \"5df5e8b8-2bc3-4788-bf77-4f32a3d52f65\") " pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:08:16.005265 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:16.005185 2531 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 13:08:16.005308 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:16.005267 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-original-pull-secret podName:5df5e8b8-2bc3-4788-bf77-4f32a3d52f65 nodeName:}" failed. No retries permitted until 2026-07-23 13:08:48.005250454 +0000 UTC m=+65.244526434 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-original-pull-secret") pod "global-pull-secret-syncer-vfgv8" (UID: "5df5e8b8-2bc3-4788-bf77-4f32a3d52f65") : object "kube-system"/"original-pull-secret" not registered Jul 23 13:08:16.106370 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:16.106321 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtbcw\" (UniqueName: \"kubernetes.io/projected/65c6347a-d5cc-4964-b5dd-b5b4098e0c5e-kube-api-access-jtbcw\") pod \"network-check-target-tc68c\" (UID: \"65c6347a-d5cc-4964-b5dd-b5b4098e0c5e\") " pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:08:16.106521 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:16.106488 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 13:08:16.106521 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:16.106510 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 13:08:16.106521 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:16.106520 2531 projected.go:196] Error preparing data for projected volume kube-api-access-jtbcw for pod openshift-network-diagnostics/network-check-target-tc68c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:08:16.106618 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:16.106574 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65c6347a-d5cc-4964-b5dd-b5b4098e0c5e-kube-api-access-jtbcw podName:65c6347a-d5cc-4964-b5dd-b5b4098e0c5e nodeName:}" failed. No retries permitted until 2026-07-23 13:08:48.106557498 +0000 UTC m=+65.345833456 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jtbcw" (UniqueName: "kubernetes.io/projected/65c6347a-d5cc-4964-b5dd-b5b4098e0c5e-kube-api-access-jtbcw") pod "network-check-target-tc68c" (UID: "65c6347a-d5cc-4964-b5dd-b5b4098e0c5e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:08:16.326720 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:16.326687 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:08:16.326879 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:16.326692 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:08:16.326879 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:16.326800 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfgv8" podUID="5df5e8b8-2bc3-4788-bf77-4f32a3d52f65" Jul 23 13:08:16.326879 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:16.326859 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc68c" podUID="65c6347a-d5cc-4964-b5dd-b5b4098e0c5e" Jul 23 13:08:17.326057 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:17.326017 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:08:17.326581 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:17.326163 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cp74b" podUID="e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3" Jul 23 13:08:17.700998 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:17.700925 2531 generic.go:356] "Generic (PLEG): container finished" podID="e6fae90e-8282-40c3-9720-4136984b837a" containerID="157256ac6fb6b10ffdd04060b8cb8ad50b12b88ee7723c13675fb935d8560430" exitCode=0 Jul 23 13:08:17.700998 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:17.700958 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwp6q" event={"ID":"e6fae90e-8282-40c3-9720-4136984b837a","Type":"ContainerDied","Data":"157256ac6fb6b10ffdd04060b8cb8ad50b12b88ee7723c13675fb935d8560430"} Jul 23 13:08:18.326907 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:18.326874 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:08:18.327393 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:18.326874 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:08:18.327393 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:18.327006 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfgv8" podUID="5df5e8b8-2bc3-4788-bf77-4f32a3d52f65" Jul 23 13:08:18.327393 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:18.327086 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc68c" podUID="65c6347a-d5cc-4964-b5dd-b5b4098e0c5e" Jul 23 13:08:19.326900 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:19.326868 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:08:19.327053 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:19.327013 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cp74b" podUID="e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3" Jul 23 13:08:20.326131 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:20.326095 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:08:20.326290 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:20.326095 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:08:20.326290 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:20.326210 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfgv8" podUID="5df5e8b8-2bc3-4788-bf77-4f32a3d52f65" Jul 23 13:08:20.326407 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:20.326286 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc68c" podUID="65c6347a-d5cc-4964-b5dd-b5b4098e0c5e" Jul 23 13:08:21.326339 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:21.326302 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:08:21.326796 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:21.326458 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cp74b" podUID="e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3" Jul 23 13:08:21.713480 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:21.713444 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwp6q" event={"ID":"e6fae90e-8282-40c3-9720-4136984b837a","Type":"ContainerStarted","Data":"50f38211bef27863673e4a08818c9c8ba2b7622cab20283b1ba9365ee1fb9e9b"} Jul 23 13:08:22.082651 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:22.082606 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vfgv8"] Jul 23 13:08:22.082810 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:22.082742 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:08:22.082868 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:22.082845 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfgv8" podUID="5df5e8b8-2bc3-4788-bf77-4f32a3d52f65" Jul 23 13:08:22.083456 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:22.083420 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tc68c"] Jul 23 13:08:22.083568 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:22.083529 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:08:22.083639 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:22.083606 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc68c" podUID="65c6347a-d5cc-4964-b5dd-b5b4098e0c5e" Jul 23 13:08:22.086175 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:22.085836 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cp74b"] Jul 23 13:08:22.086175 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:22.085921 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:08:22.086175 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:22.086031 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cp74b" podUID="e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3" Jul 23 13:08:22.719316 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:22.719238 2531 generic.go:356] "Generic (PLEG): container finished" podID="e6fae90e-8282-40c3-9720-4136984b837a" containerID="50f38211bef27863673e4a08818c9c8ba2b7622cab20283b1ba9365ee1fb9e9b" exitCode=0 Jul 23 13:08:22.719316 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:22.719286 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwp6q" event={"ID":"e6fae90e-8282-40c3-9720-4136984b837a","Type":"ContainerDied","Data":"50f38211bef27863673e4a08818c9c8ba2b7622cab20283b1ba9365ee1fb9e9b"} Jul 23 13:08:23.327133 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:23.326966 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:08:23.327286 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:23.327194 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cp74b" podUID="e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3" Jul 23 13:08:23.724816 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:23.724731 2531 generic.go:356] "Generic (PLEG): container finished" podID="e6fae90e-8282-40c3-9720-4136984b837a" containerID="45daa2ebc8788eccbfbc46666e211df5317dfb373936474effc6c47c5dc4d0d3" exitCode=0 Jul 23 13:08:23.724816 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:23.724808 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwp6q" event={"ID":"e6fae90e-8282-40c3-9720-4136984b837a","Type":"ContainerDied","Data":"45daa2ebc8788eccbfbc46666e211df5317dfb373936474effc6c47c5dc4d0d3"} Jul 23 13:08:24.326601 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:24.326570 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:08:24.326601 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:24.326583 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:08:24.326875 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:24.326668 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfgv8" podUID="5df5e8b8-2bc3-4788-bf77-4f32a3d52f65" Jul 23 13:08:24.326875 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:24.326778 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc68c" podUID="65c6347a-d5cc-4964-b5dd-b5b4098e0c5e" Jul 23 13:08:24.730202 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:24.730160 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwp6q" event={"ID":"e6fae90e-8282-40c3-9720-4136984b837a","Type":"ContainerStarted","Data":"3a05716392754bab64a5d0a7f22476bce2f4ee181f1201aed2eb92d5a2186b61"} Jul 23 13:08:24.754092 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:24.754044 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zwp6q" podStartSLOduration=7.780958553 podStartE2EDuration="40.754031328s" totalImagesPullingTime="32.973072775s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:07:44 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:08:24.752824812 +0000 UTC m=+41.992100789" watchObservedRunningTime="2026-07-23 13:08:24.754031328 +0000 UTC m=+41.993307304" Jul 23 13:08:25.326836 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:25.326800 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:08:25.327007 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:25.326958 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cp74b" podUID="e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3" Jul 23 13:08:26.326552 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.326522 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:08:26.326983 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.326563 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:08:26.326983 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:26.326662 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc68c" podUID="65c6347a-d5cc-4964-b5dd-b5b4098e0c5e" Jul 23 13:08:26.326983 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:26.326780 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vfgv8" podUID="5df5e8b8-2bc3-4788-bf77-4f32a3d52f65" Jul 23 13:08:26.593067 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.593038 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-1.ec2.internal" event="NodeReady" Jul 23 13:08:26.593203 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.593157 2531 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 23 13:08:26.629625 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.629595 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7d4fcff47f-52ds5"] Jul 23 13:08:26.664445 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.664419 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cqn57"] Jul 23 13:08:26.664599 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.664547 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.666815 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.666792 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 23 13:08:26.667055 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.667036 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 23 13:08:26.667143 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.667058 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9cnr7\"" Jul 23 13:08:26.667143 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.667039 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 23 13:08:26.675486 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.674967 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 23 13:08:26.680164 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.680146 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5zfnk"] Jul 23 13:08:26.680261 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.680247 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cqn57" Jul 23 13:08:26.682692 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.682668 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 23 13:08:26.683005 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.682990 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 23 13:08:26.683832 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.683785 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 23 13:08:26.684007 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.683993 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-njgx6\"" Jul 23 13:08:26.684056 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.683993 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 23 13:08:26.702272 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.702251 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d4fcff47f-52ds5"] Jul 23 13:08:26.702398 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.702279 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cqn57"] Jul 23 13:08:26.702398 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.702302 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5zfnk"] Jul 23 13:08:26.702517 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.702397 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5zfnk" Jul 23 13:08:26.704666 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.704634 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 23 13:08:26.704666 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.704642 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 23 13:08:26.704823 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.704798 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-b6wbz\"" Jul 23 13:08:26.744496 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.744468 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-cd2jt"] Jul 23 13:08:26.766226 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.766204 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cd2jt"] Jul 23 13:08:26.766341 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.766287 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cd2jt" Jul 23 13:08:26.771302 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.771278 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 23 13:08:26.771451 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.771305 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 23 13:08:26.771451 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.771316 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mbwhb\"" Jul 23 13:08:26.775809 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.775791 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 23 13:08:26.788993 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.788971 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/459f4b73-abfd-426c-abcd-7de9467a3fab-registry-tls\") pod \"image-registry-7d4fcff47f-52ds5\" (UID: \"459f4b73-abfd-426c-abcd-7de9467a3fab\") " pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.789110 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.789005 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/459f4b73-abfd-426c-abcd-7de9467a3fab-ca-trust-extracted\") pod \"image-registry-7d4fcff47f-52ds5\" (UID: \"459f4b73-abfd-426c-abcd-7de9467a3fab\") " pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.789110 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.789034 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/459f4b73-abfd-426c-abcd-7de9467a3fab-trusted-ca\") pod \"image-registry-7d4fcff47f-52ds5\" (UID: \"459f4b73-abfd-426c-abcd-7de9467a3fab\") " pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.789270 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.789123 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/459f4b73-abfd-426c-abcd-7de9467a3fab-image-registry-private-configuration\") pod \"image-registry-7d4fcff47f-52ds5\" (UID: \"459f4b73-abfd-426c-abcd-7de9467a3fab\") " pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.789270 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.789151 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/459f4b73-abfd-426c-abcd-7de9467a3fab-registry-certificates\") pod \"image-registry-7d4fcff47f-52ds5\" (UID: \"459f4b73-abfd-426c-abcd-7de9467a3fab\") " pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.789270 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.789168 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e39a3e4d-f59e-4de3-b64d-3924df816483-crio-socket\") pod \"insights-runtime-extractor-cqn57\" (UID: \"e39a3e4d-f59e-4de3-b64d-3924df816483\") " pod="openshift-insights/insights-runtime-extractor-cqn57" Jul 23 13:08:26.789270 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.789209 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e39a3e4d-f59e-4de3-b64d-3924df816483-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cqn57\" (UID: \"e39a3e4d-f59e-4de3-b64d-3924df816483\") " pod="openshift-insights/insights-runtime-extractor-cqn57" Jul 23 13:08:26.789270 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.789247 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e39a3e4d-f59e-4de3-b64d-3924df816483-data-volume\") pod \"insights-runtime-extractor-cqn57\" (UID: \"e39a3e4d-f59e-4de3-b64d-3924df816483\") " pod="openshift-insights/insights-runtime-extractor-cqn57" Jul 23 13:08:26.789544 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.789283 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e39a3e4d-f59e-4de3-b64d-3924df816483-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cqn57\" (UID: \"e39a3e4d-f59e-4de3-b64d-3924df816483\") " pod="openshift-insights/insights-runtime-extractor-cqn57" Jul 23 13:08:26.789544 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.789305 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x44fg\" (UniqueName: \"kubernetes.io/projected/e39a3e4d-f59e-4de3-b64d-3924df816483-kube-api-access-x44fg\") pod \"insights-runtime-extractor-cqn57\" (UID: \"e39a3e4d-f59e-4de3-b64d-3924df816483\") " pod="openshift-insights/insights-runtime-extractor-cqn57" Jul 23 13:08:26.789544 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.789321 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/459f4b73-abfd-426c-abcd-7de9467a3fab-installation-pull-secrets\") pod \"image-registry-7d4fcff47f-52ds5\" (UID: \"459f4b73-abfd-426c-abcd-7de9467a3fab\") " pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.789544 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.789377 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/459f4b73-abfd-426c-abcd-7de9467a3fab-bound-sa-token\") pod \"image-registry-7d4fcff47f-52ds5\" (UID: \"459f4b73-abfd-426c-abcd-7de9467a3fab\") " pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.789544 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.789410 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlntc\" (UniqueName: \"kubernetes.io/projected/459f4b73-abfd-426c-abcd-7de9467a3fab-kube-api-access-wlntc\") pod \"image-registry-7d4fcff47f-52ds5\" (UID: \"459f4b73-abfd-426c-abcd-7de9467a3fab\") " pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.890054 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.889965 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx7rj\" (UniqueName: \"kubernetes.io/projected/b50baffc-9fe0-4ba8-9292-ca620a946b89-kube-api-access-kx7rj\") pod \"dns-default-5zfnk\" (UID: \"b50baffc-9fe0-4ba8-9292-ca620a946b89\") " pod="openshift-dns/dns-default-5zfnk" Jul 23 13:08:26.890054 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.890010 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b50baffc-9fe0-4ba8-9292-ca620a946b89-metrics-tls\") pod \"dns-default-5zfnk\" (UID: \"b50baffc-9fe0-4ba8-9292-ca620a946b89\") " pod="openshift-dns/dns-default-5zfnk" Jul 23 13:08:26.890251 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.890067 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/459f4b73-abfd-426c-abcd-7de9467a3fab-registry-tls\") pod \"image-registry-7d4fcff47f-52ds5\" (UID: \"459f4b73-abfd-426c-abcd-7de9467a3fab\") " pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.890251 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.890098 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/459f4b73-abfd-426c-abcd-7de9467a3fab-ca-trust-extracted\") pod \"image-registry-7d4fcff47f-52ds5\" (UID: \"459f4b73-abfd-426c-abcd-7de9467a3fab\") " pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.890251 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.890203 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/459f4b73-abfd-426c-abcd-7de9467a3fab-trusted-ca\") pod \"image-registry-7d4fcff47f-52ds5\" (UID: \"459f4b73-abfd-426c-abcd-7de9467a3fab\") " pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.890445 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.890273 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/459f4b73-abfd-426c-abcd-7de9467a3fab-image-registry-private-configuration\") pod \"image-registry-7d4fcff47f-52ds5\" (UID: \"459f4b73-abfd-426c-abcd-7de9467a3fab\") " pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.890445 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.890304 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/459f4b73-abfd-426c-abcd-7de9467a3fab-registry-certificates\") pod \"image-registry-7d4fcff47f-52ds5\" (UID: \"459f4b73-abfd-426c-abcd-7de9467a3fab\") " pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.890445 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.890328 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e39a3e4d-f59e-4de3-b64d-3924df816483-crio-socket\") pod \"insights-runtime-extractor-cqn57\" (UID: \"e39a3e4d-f59e-4de3-b64d-3924df816483\") " pod="openshift-insights/insights-runtime-extractor-cqn57" Jul 23 13:08:26.890573 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.890463 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/459f4b73-abfd-426c-abcd-7de9467a3fab-ca-trust-extracted\") pod \"image-registry-7d4fcff47f-52ds5\" (UID: \"459f4b73-abfd-426c-abcd-7de9467a3fab\") " pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.890625 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.890570 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e39a3e4d-f59e-4de3-b64d-3924df816483-crio-socket\") pod \"insights-runtime-extractor-cqn57\" (UID: \"e39a3e4d-f59e-4de3-b64d-3924df816483\") " pod="openshift-insights/insights-runtime-extractor-cqn57" Jul 23 13:08:26.890625 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.890383 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e39a3e4d-f59e-4de3-b64d-3924df816483-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cqn57\" (UID: \"e39a3e4d-f59e-4de3-b64d-3924df816483\") " pod="openshift-insights/insights-runtime-extractor-cqn57" Jul 23 13:08:26.890714 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.890640 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b50baffc-9fe0-4ba8-9292-ca620a946b89-config-volume\") pod \"dns-default-5zfnk\" (UID: \"b50baffc-9fe0-4ba8-9292-ca620a946b89\") " pod="openshift-dns/dns-default-5zfnk" Jul 23 13:08:26.890714 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.890684 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e39a3e4d-f59e-4de3-b64d-3924df816483-data-volume\") pod \"insights-runtime-extractor-cqn57\" (UID: \"e39a3e4d-f59e-4de3-b64d-3924df816483\") " pod="openshift-insights/insights-runtime-extractor-cqn57" Jul 23 13:08:26.890815 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.890712 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e39a3e4d-f59e-4de3-b64d-3924df816483-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cqn57\" (UID: \"e39a3e4d-f59e-4de3-b64d-3924df816483\") " pod="openshift-insights/insights-runtime-extractor-cqn57" Jul 23 13:08:26.890815 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.890737 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x44fg\" (UniqueName: \"kubernetes.io/projected/e39a3e4d-f59e-4de3-b64d-3924df816483-kube-api-access-x44fg\") pod \"insights-runtime-extractor-cqn57\" (UID: \"e39a3e4d-f59e-4de3-b64d-3924df816483\") " pod="openshift-insights/insights-runtime-extractor-cqn57" Jul 23 13:08:26.890815 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.890764 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/459f4b73-abfd-426c-abcd-7de9467a3fab-installation-pull-secrets\") pod \"image-registry-7d4fcff47f-52ds5\" (UID: \"459f4b73-abfd-426c-abcd-7de9467a3fab\") " pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.890815 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.890809 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/459f4b73-abfd-426c-abcd-7de9467a3fab-bound-sa-token\") pod \"image-registry-7d4fcff47f-52ds5\" (UID: \"459f4b73-abfd-426c-abcd-7de9467a3fab\") " pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.891001 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.890837 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghpps\" (UniqueName: \"kubernetes.io/projected/8261ecc7-323a-4dd7-8a6a-43381de92427-kube-api-access-ghpps\") pod \"ingress-canary-cd2jt\" (UID: \"8261ecc7-323a-4dd7-8a6a-43381de92427\") " pod="openshift-ingress-canary/ingress-canary-cd2jt" Jul 23 13:08:26.891001 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.890870 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlntc\" (UniqueName: \"kubernetes.io/projected/459f4b73-abfd-426c-abcd-7de9467a3fab-kube-api-access-wlntc\") pod \"image-registry-7d4fcff47f-52ds5\" (UID: \"459f4b73-abfd-426c-abcd-7de9467a3fab\") " pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.891001 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.890895 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8261ecc7-323a-4dd7-8a6a-43381de92427-cert\") pod \"ingress-canary-cd2jt\" (UID: \"8261ecc7-323a-4dd7-8a6a-43381de92427\") " pod="openshift-ingress-canary/ingress-canary-cd2jt" Jul 23 13:08:26.891001 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.890924 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b50baffc-9fe0-4ba8-9292-ca620a946b89-tmp-dir\") pod \"dns-default-5zfnk\" (UID: \"b50baffc-9fe0-4ba8-9292-ca620a946b89\") " pod="openshift-dns/dns-default-5zfnk" Jul 23 13:08:26.891001 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.890979 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e39a3e4d-f59e-4de3-b64d-3924df816483-data-volume\") pod \"insights-runtime-extractor-cqn57\" (UID: \"e39a3e4d-f59e-4de3-b64d-3924df816483\") " pod="openshift-insights/insights-runtime-extractor-cqn57" Jul 23 13:08:26.891238 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.891089 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e39a3e4d-f59e-4de3-b64d-3924df816483-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cqn57\" (UID: \"e39a3e4d-f59e-4de3-b64d-3924df816483\") " pod="openshift-insights/insights-runtime-extractor-cqn57" Jul 23 13:08:26.891238 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.891131 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/459f4b73-abfd-426c-abcd-7de9467a3fab-registry-certificates\") pod \"image-registry-7d4fcff47f-52ds5\" (UID: \"459f4b73-abfd-426c-abcd-7de9467a3fab\") " pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.891238 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.891182 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/459f4b73-abfd-426c-abcd-7de9467a3fab-trusted-ca\") pod \"image-registry-7d4fcff47f-52ds5\" (UID: \"459f4b73-abfd-426c-abcd-7de9467a3fab\") " pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.894473 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.894440 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e39a3e4d-f59e-4de3-b64d-3924df816483-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cqn57\" (UID: \"e39a3e4d-f59e-4de3-b64d-3924df816483\") " pod="openshift-insights/insights-runtime-extractor-cqn57" Jul 23 13:08:26.894578 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.894531 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/459f4b73-abfd-426c-abcd-7de9467a3fab-image-registry-private-configuration\") pod \"image-registry-7d4fcff47f-52ds5\" (UID: \"459f4b73-abfd-426c-abcd-7de9467a3fab\") " pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.894619 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.894574 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/459f4b73-abfd-426c-abcd-7de9467a3fab-installation-pull-secrets\") pod \"image-registry-7d4fcff47f-52ds5\" (UID: \"459f4b73-abfd-426c-abcd-7de9467a3fab\") " pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.894666 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.894651 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/459f4b73-abfd-426c-abcd-7de9467a3fab-registry-tls\") pod \"image-registry-7d4fcff47f-52ds5\" (UID: \"459f4b73-abfd-426c-abcd-7de9467a3fab\") " pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.901245 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.901219 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlntc\" (UniqueName: \"kubernetes.io/projected/459f4b73-abfd-426c-abcd-7de9467a3fab-kube-api-access-wlntc\") pod \"image-registry-7d4fcff47f-52ds5\" (UID: \"459f4b73-abfd-426c-abcd-7de9467a3fab\") " pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.901814 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.901797 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/459f4b73-abfd-426c-abcd-7de9467a3fab-bound-sa-token\") pod \"image-registry-7d4fcff47f-52ds5\" (UID: \"459f4b73-abfd-426c-abcd-7de9467a3fab\") " pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.901956 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.901938 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x44fg\" (UniqueName: \"kubernetes.io/projected/e39a3e4d-f59e-4de3-b64d-3924df816483-kube-api-access-x44fg\") pod \"insights-runtime-extractor-cqn57\" (UID: \"e39a3e4d-f59e-4de3-b64d-3924df816483\") " pod="openshift-insights/insights-runtime-extractor-cqn57" Jul 23 13:08:26.977139 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.977107 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:26.989260 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.989231 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cqn57" Jul 23 13:08:26.991521 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.991503 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b50baffc-9fe0-4ba8-9292-ca620a946b89-config-volume\") pod \"dns-default-5zfnk\" (UID: \"b50baffc-9fe0-4ba8-9292-ca620a946b89\") " pod="openshift-dns/dns-default-5zfnk" Jul 23 13:08:26.991575 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.991555 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghpps\" (UniqueName: \"kubernetes.io/projected/8261ecc7-323a-4dd7-8a6a-43381de92427-kube-api-access-ghpps\") pod \"ingress-canary-cd2jt\" (UID: \"8261ecc7-323a-4dd7-8a6a-43381de92427\") " pod="openshift-ingress-canary/ingress-canary-cd2jt" Jul 23 13:08:26.991609 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.991584 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8261ecc7-323a-4dd7-8a6a-43381de92427-cert\") pod \"ingress-canary-cd2jt\" (UID: \"8261ecc7-323a-4dd7-8a6a-43381de92427\") " pod="openshift-ingress-canary/ingress-canary-cd2jt" Jul 23 13:08:26.991609 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.991603 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b50baffc-9fe0-4ba8-9292-ca620a946b89-tmp-dir\") pod \"dns-default-5zfnk\" (UID: \"b50baffc-9fe0-4ba8-9292-ca620a946b89\") " pod="openshift-dns/dns-default-5zfnk" Jul 23 13:08:26.991677 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.991620 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kx7rj\" (UniqueName: \"kubernetes.io/projected/b50baffc-9fe0-4ba8-9292-ca620a946b89-kube-api-access-kx7rj\") pod \"dns-default-5zfnk\" (UID: \"b50baffc-9fe0-4ba8-9292-ca620a946b89\") " pod="openshift-dns/dns-default-5zfnk" Jul 23 13:08:26.991677 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.991652 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b50baffc-9fe0-4ba8-9292-ca620a946b89-metrics-tls\") pod \"dns-default-5zfnk\" (UID: \"b50baffc-9fe0-4ba8-9292-ca620a946b89\") " pod="openshift-dns/dns-default-5zfnk" Jul 23 13:08:26.992045 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.992017 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b50baffc-9fe0-4ba8-9292-ca620a946b89-tmp-dir\") pod \"dns-default-5zfnk\" (UID: \"b50baffc-9fe0-4ba8-9292-ca620a946b89\") " pod="openshift-dns/dns-default-5zfnk" Jul 23 13:08:26.992131 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.992070 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b50baffc-9fe0-4ba8-9292-ca620a946b89-config-volume\") pod \"dns-default-5zfnk\" (UID: \"b50baffc-9fe0-4ba8-9292-ca620a946b89\") " pod="openshift-dns/dns-default-5zfnk" Jul 23 13:08:26.994145 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.994122 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b50baffc-9fe0-4ba8-9292-ca620a946b89-metrics-tls\") pod \"dns-default-5zfnk\" (UID: \"b50baffc-9fe0-4ba8-9292-ca620a946b89\") " pod="openshift-dns/dns-default-5zfnk" Jul 23 13:08:26.994225 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.994126 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8261ecc7-323a-4dd7-8a6a-43381de92427-cert\") pod \"ingress-canary-cd2jt\" (UID: \"8261ecc7-323a-4dd7-8a6a-43381de92427\") " pod="openshift-ingress-canary/ingress-canary-cd2jt" Jul 23 13:08:26.999671 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.999653 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghpps\" (UniqueName: \"kubernetes.io/projected/8261ecc7-323a-4dd7-8a6a-43381de92427-kube-api-access-ghpps\") pod \"ingress-canary-cd2jt\" (UID: \"8261ecc7-323a-4dd7-8a6a-43381de92427\") " pod="openshift-ingress-canary/ingress-canary-cd2jt" Jul 23 13:08:26.999931 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:26.999912 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx7rj\" (UniqueName: \"kubernetes.io/projected/b50baffc-9fe0-4ba8-9292-ca620a946b89-kube-api-access-kx7rj\") pod \"dns-default-5zfnk\" (UID: \"b50baffc-9fe0-4ba8-9292-ca620a946b89\") " pod="openshift-dns/dns-default-5zfnk" Jul 23 13:08:27.010154 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:27.010136 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5zfnk" Jul 23 13:08:27.074837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:27.074684 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cd2jt" Jul 23 13:08:27.211453 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:27.211257 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d4fcff47f-52ds5"] Jul 23 13:08:27.217778 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:27.217754 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cqn57"] Jul 23 13:08:27.219473 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:08:27.219427 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod459f4b73_abfd_426c_abcd_7de9467a3fab.slice/crio-87f7370033a0adfab07e99d9b6b73ec34d5c1c17503edadeffa8af370fd14a80 WatchSource:0}: Error finding container 87f7370033a0adfab07e99d9b6b73ec34d5c1c17503edadeffa8af370fd14a80: Status 404 returned error can't find the container with id 87f7370033a0adfab07e99d9b6b73ec34d5c1c17503edadeffa8af370fd14a80 Jul 23 13:08:27.222556 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:08:27.222510 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode39a3e4d_f59e_4de3_b64d_3924df816483.slice/crio-5a4453323e9056acf24af32bc718a4533c864ba6191b13c87648a8ea3c824b87 WatchSource:0}: Error finding container 5a4453323e9056acf24af32bc718a4533c864ba6191b13c87648a8ea3c824b87: Status 404 returned error can't find the container with id 5a4453323e9056acf24af32bc718a4533c864ba6191b13c87648a8ea3c824b87 Jul 23 13:08:27.234150 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:27.234126 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5zfnk"] Jul 23 13:08:27.237244 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:08:27.237222 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb50baffc_9fe0_4ba8_9292_ca620a946b89.slice/crio-3a239207e81d2d1c446cea06824f7fe92aec3a098ce23406fb3d2a871c36eeea WatchSource:0}: Error finding container 3a239207e81d2d1c446cea06824f7fe92aec3a098ce23406fb3d2a871c36eeea: Status 404 returned error can't find the container with id 3a239207e81d2d1c446cea06824f7fe92aec3a098ce23406fb3d2a871c36eeea Jul 23 13:08:27.255802 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:27.255769 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cd2jt"] Jul 23 13:08:27.264236 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:08:27.264208 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8261ecc7_323a_4dd7_8a6a_43381de92427.slice/crio-dab009d06881d0d6f340b6c63b6717dddcd3a668ad69cca05255c3478b134ac9 WatchSource:0}: Error finding container dab009d06881d0d6f340b6c63b6717dddcd3a668ad69cca05255c3478b134ac9: Status 404 returned error can't find the container with id dab009d06881d0d6f340b6c63b6717dddcd3a668ad69cca05255c3478b134ac9 Jul 23 13:08:27.326801 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:27.326782 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:08:27.329184 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:27.329158 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 13:08:27.329289 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:27.329213 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z5gct\"" Jul 23 13:08:27.738841 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:27.738803 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cd2jt" event={"ID":"8261ecc7-323a-4dd7-8a6a-43381de92427","Type":"ContainerStarted","Data":"dab009d06881d0d6f340b6c63b6717dddcd3a668ad69cca05255c3478b134ac9"} Jul 23 13:08:27.740252 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:27.740218 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5zfnk" event={"ID":"b50baffc-9fe0-4ba8-9292-ca620a946b89","Type":"ContainerStarted","Data":"3a239207e81d2d1c446cea06824f7fe92aec3a098ce23406fb3d2a871c36eeea"} Jul 23 13:08:27.741727 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:27.741695 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" event={"ID":"459f4b73-abfd-426c-abcd-7de9467a3fab","Type":"ContainerStarted","Data":"c1d2189d48fc0deabb3d68389dd7a1249c254561517a37f3ffcb6a05cd2ab22a"} Jul 23 13:08:27.741823 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:27.741741 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" event={"ID":"459f4b73-abfd-426c-abcd-7de9467a3fab","Type":"ContainerStarted","Data":"87f7370033a0adfab07e99d9b6b73ec34d5c1c17503edadeffa8af370fd14a80"} Jul 23 13:08:27.741823 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:27.741780 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:27.742856 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:27.742834 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cqn57" event={"ID":"e39a3e4d-f59e-4de3-b64d-3924df816483","Type":"ContainerStarted","Data":"5a4453323e9056acf24af32bc718a4533c864ba6191b13c87648a8ea3c824b87"} Jul 23 13:08:27.760876 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:27.760120 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" podStartSLOduration=7.760100415 podStartE2EDuration="7.760100415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:08:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:08:27.759580132 +0000 UTC m=+44.998856110" watchObservedRunningTime="2026-07-23 13:08:27.760100415 +0000 UTC m=+44.999376371" Jul 23 13:08:28.326602 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:28.326566 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:08:28.328018 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:28.327481 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:08:28.329283 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:28.329259 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 13:08:28.330016 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:28.329994 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 13:08:28.330240 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:28.330212 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xq2s8\"" Jul 23 13:08:28.330478 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:28.330448 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 13:08:34.765322 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:34.765075 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cd2jt" event={"ID":"8261ecc7-323a-4dd7-8a6a-43381de92427","Type":"ContainerStarted","Data":"4423ba60e8c568e45578ee21f19ab4f55068e7235ce40af7bc1e73d6ea69d09f"} Jul 23 13:08:34.766865 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:34.766838 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5zfnk" event={"ID":"b50baffc-9fe0-4ba8-9292-ca620a946b89","Type":"ContainerStarted","Data":"1485d55f265b371ebfd5b59291ef280d0e460257f5b47494bde571dfa6b4ca80"} Jul 23 13:08:34.766986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:34.766889 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5zfnk" event={"ID":"b50baffc-9fe0-4ba8-9292-ca620a946b89","Type":"ContainerStarted","Data":"e14d7394637929740298123199fc0f16edb783c748dff05077903ec4ebb6cb1e"} Jul 23 13:08:34.766986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:34.766917 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5zfnk" Jul 23 13:08:34.768195 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:34.768176 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cqn57" event={"ID":"e39a3e4d-f59e-4de3-b64d-3924df816483","Type":"ContainerStarted","Data":"9395546ff8b7a8bc55e5ba1647cd893ba4f9c0de91a149434cf707c5e56acaff"} Jul 23 13:08:34.781237 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:34.781191 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-cd2jt" podStartSLOduration=2.099635741 podStartE2EDuration="8.781174427s" totalImagesPullingTime="6.681538686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:08:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:08:34.78098496 +0000 UTC m=+52.020260936" watchObservedRunningTime="2026-07-23 13:08:34.781174427 +0000 UTC m=+52.020450403" Jul 23 13:08:34.796677 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:34.796631 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5zfnk" podStartSLOduration=2.088427349 podStartE2EDuration="8.796615192s" totalImagesPullingTime="6.708187843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:08:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:08:34.796059293 +0000 UTC m=+52.035335268" watchObservedRunningTime="2026-07-23 13:08:34.796615192 +0000 UTC m=+52.035891206" Jul 23 13:08:38.783092 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:38.782860 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cqn57" event={"ID":"e39a3e4d-f59e-4de3-b64d-3924df816483","Type":"ContainerStarted","Data":"d8f5d56f16001736851c8ca927599bcb9cded18588bcc4d26363c11e5f60ed61"} Jul 23 13:08:43.800473 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:43.800382 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cqn57" event={"ID":"e39a3e4d-f59e-4de3-b64d-3924df816483","Type":"ContainerStarted","Data":"7ba889688c18bafc060884b0a1c3ae9f440ed7924d8143f1cdb1d5ac95b3ef72"} Jul 23 13:08:43.818282 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:43.818232 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cqn57" podStartSLOduration=1.877662618 podStartE2EDuration="17.818218212s" totalImagesPullingTime="15.940555594s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:08:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:08:43.816858661 +0000 UTC m=+61.056134637" watchObservedRunningTime="2026-07-23 13:08:43.818218212 +0000 UTC m=+61.057494188" Jul 23 13:08:44.701498 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:44.701462 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t6m8j" Jul 23 13:08:44.774001 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:44.773969 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5zfnk" Jul 23 13:08:46.981564 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:46.981528 2531 patch_prober.go:28] interesting pod/image-registry-7d4fcff47f-52ds5 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 23 13:08:46.981929 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:46.981583 2531 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" podUID="459f4b73-abfd-426c-abcd-7de9467a3fab" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 13:08:47.945289 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:47.945246 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-metrics-certs\") pod \"network-metrics-daemon-cp74b\" (UID: \"e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3\") " pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:08:47.947749 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:47.947731 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 13:08:47.960294 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:47.960269 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3-metrics-certs\") pod \"network-metrics-daemon-cp74b\" (UID: \"e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3\") " pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:08:48.038231 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:48.038198 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z5gct\"" Jul 23 13:08:48.046165 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:48.046144 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cp74b" Jul 23 13:08:48.046372 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:48.046330 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-original-pull-secret\") pod \"global-pull-secret-syncer-vfgv8\" (UID: \"5df5e8b8-2bc3-4788-bf77-4f32a3d52f65\") " pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:08:48.048698 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:48.048679 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 13:08:48.059780 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:48.059754 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5df5e8b8-2bc3-4788-bf77-4f32a3d52f65-original-pull-secret\") pod \"global-pull-secret-syncer-vfgv8\" (UID: \"5df5e8b8-2bc3-4788-bf77-4f32a3d52f65\") " pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:08:48.147152 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:48.147120 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtbcw\" (UniqueName: \"kubernetes.io/projected/65c6347a-d5cc-4964-b5dd-b5b4098e0c5e-kube-api-access-jtbcw\") pod \"network-check-target-tc68c\" (UID: \"65c6347a-d5cc-4964-b5dd-b5b4098e0c5e\") " pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:08:48.149943 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:48.149920 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vfgv8" Jul 23 13:08:48.150060 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:48.150038 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 13:08:48.160458 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:48.160433 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 13:08:48.170573 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:48.170553 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtbcw\" (UniqueName: \"kubernetes.io/projected/65c6347a-d5cc-4964-b5dd-b5b4098e0c5e-kube-api-access-jtbcw\") pod \"network-check-target-tc68c\" (UID: \"65c6347a-d5cc-4964-b5dd-b5b4098e0c5e\") " pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:08:48.175037 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:48.175011 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cp74b"] Jul 23 13:08:48.178648 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:08:48.178612 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0ceb6d6_8b7c_4e99_9d9e_16c3493425a3.slice/crio-fa631947497180ba07ad44ae935092cd14e8577331ffe60d254501f4b4072065 WatchSource:0}: Error finding container fa631947497180ba07ad44ae935092cd14e8577331ffe60d254501f4b4072065: Status 404 returned error can't find the container with id fa631947497180ba07ad44ae935092cd14e8577331ffe60d254501f4b4072065 Jul 23 13:08:48.273623 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:48.273592 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vfgv8"] Jul 23 13:08:48.276280 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:08:48.276246 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5df5e8b8_2bc3_4788_bf77_4f32a3d52f65.slice/crio-62cda79be80ceb98b1bedda509bc512bdf76f72ada54973501d6e8b210a208bc WatchSource:0}: Error finding container 62cda79be80ceb98b1bedda509bc512bdf76f72ada54973501d6e8b210a208bc: Status 404 returned error can't find the container with id 62cda79be80ceb98b1bedda509bc512bdf76f72ada54973501d6e8b210a208bc Jul 23 13:08:48.446531 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:48.446504 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xq2s8\"" Jul 23 13:08:48.454425 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:48.454373 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:08:48.575658 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:48.575628 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tc68c"] Jul 23 13:08:48.578473 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:08:48.578448 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65c6347a_d5cc_4964_b5dd_b5b4098e0c5e.slice/crio-554448a0cc60ade96eff43676b86093cba2fac175944bd4d3e5097ea3dcb6d3f WatchSource:0}: Error finding container 554448a0cc60ade96eff43676b86093cba2fac175944bd4d3e5097ea3dcb6d3f: Status 404 returned error can't find the container with id 554448a0cc60ade96eff43676b86093cba2fac175944bd4d3e5097ea3dcb6d3f Jul 23 13:08:48.755536 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:48.755459 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7d4fcff47f-52ds5" Jul 23 13:08:48.822430 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:48.822394 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tc68c" event={"ID":"65c6347a-d5cc-4964-b5dd-b5b4098e0c5e","Type":"ContainerStarted","Data":"554448a0cc60ade96eff43676b86093cba2fac175944bd4d3e5097ea3dcb6d3f"} Jul 23 13:08:48.824672 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:48.824642 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vfgv8" event={"ID":"5df5e8b8-2bc3-4788-bf77-4f32a3d52f65","Type":"ContainerStarted","Data":"62cda79be80ceb98b1bedda509bc512bdf76f72ada54973501d6e8b210a208bc"} Jul 23 13:08:48.825958 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:48.825933 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cp74b" event={"ID":"e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3","Type":"ContainerStarted","Data":"fa631947497180ba07ad44ae935092cd14e8577331ffe60d254501f4b4072065"} Jul 23 13:08:49.832226 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:49.832172 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cp74b" event={"ID":"e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3","Type":"ContainerStarted","Data":"573b091214076fe60c56ef1afe654f76b18be0e13c444ae27f63d750b1ae5e0f"} Jul 23 13:08:49.832226 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:49.832225 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cp74b" event={"ID":"e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3","Type":"ContainerStarted","Data":"417808f5e9b29b5ab93bef92f05c937dd588fd8c38db541f68f6773bdc6dc1f0"} Jul 23 13:08:49.847695 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:49.847636 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cp74b" podStartSLOduration=64.833324404 podStartE2EDuration="1m5.847615029s" totalImagesPullingTime="1.014290625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:07:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:08:49.846665111 +0000 UTC m=+67.085941088" watchObservedRunningTime="2026-07-23 13:08:49.847615029 +0000 UTC m=+67.086891007" Jul 23 13:08:51.552082 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.552042 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-mfg7x"] Jul 23 13:08:51.554549 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.554527 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cbk6z"] Jul 23 13:08:51.554717 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.554683 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-mfg7x" Jul 23 13:08:51.556887 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.556858 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.557130 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.557110 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 23 13:08:51.557315 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.557281 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 23 13:08:51.557481 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.557444 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-7tzw9\"" Jul 23 13:08:51.557481 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.557446 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 23 13:08:51.557481 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.557467 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 23 13:08:51.557938 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.557921 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 23 13:08:51.558077 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.557985 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 23 13:08:51.558810 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.558791 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 23 13:08:51.558944 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.558926 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 23 13:08:51.563675 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.559700 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 23 13:08:51.563675 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.559944 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wkhqp\"" Jul 23 13:08:51.569521 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.569092 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-mfg7x"] Jul 23 13:08:51.572905 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.572881 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce140f37-9913-4f41-8522-377c123c7d65-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-mfg7x\" (UID: \"ce140f37-9913-4f41-8522-377c123c7d65\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mfg7x" Jul 23 13:08:51.573004 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.572921 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvxxz\" (UniqueName: \"kubernetes.io/projected/ce140f37-9913-4f41-8522-377c123c7d65-kube-api-access-xvxxz\") pod \"kube-state-metrics-56c5594674-mfg7x\" (UID: \"ce140f37-9913-4f41-8522-377c123c7d65\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mfg7x" Jul 23 13:08:51.573004 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.572962 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ce140f37-9913-4f41-8522-377c123c7d65-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-mfg7x\" (UID: \"ce140f37-9913-4f41-8522-377c123c7d65\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mfg7x" Jul 23 13:08:51.573106 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.573020 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ce140f37-9913-4f41-8522-377c123c7d65-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-mfg7x\" (UID: \"ce140f37-9913-4f41-8522-377c123c7d65\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mfg7x" Jul 23 13:08:51.573106 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.573050 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce140f37-9913-4f41-8522-377c123c7d65-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-mfg7x\" (UID: \"ce140f37-9913-4f41-8522-377c123c7d65\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mfg7x" Jul 23 13:08:51.573262 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.573132 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce140f37-9913-4f41-8522-377c123c7d65-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-mfg7x\" (UID: \"ce140f37-9913-4f41-8522-377c123c7d65\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mfg7x" Jul 23 13:08:51.674520 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.674484 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvxxz\" (UniqueName: \"kubernetes.io/projected/ce140f37-9913-4f41-8522-377c123c7d65-kube-api-access-xvxxz\") pod \"kube-state-metrics-56c5594674-mfg7x\" (UID: \"ce140f37-9913-4f41-8522-377c123c7d65\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mfg7x" Jul 23 13:08:51.674716 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.674545 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aff84e73-559f-4d6f-a690-c38b353cf763-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.674716 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.674575 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aff84e73-559f-4d6f-a690-c38b353cf763-sys\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.674716 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.674629 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ce140f37-9913-4f41-8522-377c123c7d65-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-mfg7x\" (UID: \"ce140f37-9913-4f41-8522-377c123c7d65\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mfg7x" Jul 23 13:08:51.674716 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.674665 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ce140f37-9913-4f41-8522-377c123c7d65-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-mfg7x\" (UID: \"ce140f37-9913-4f41-8522-377c123c7d65\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mfg7x" Jul 23 13:08:51.674716 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.674711 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prl4j\" (UniqueName: \"kubernetes.io/projected/aff84e73-559f-4d6f-a690-c38b353cf763-kube-api-access-prl4j\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.674962 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.674753 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aff84e73-559f-4d6f-a690-c38b353cf763-metrics-client-ca\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.674962 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.674776 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aff84e73-559f-4d6f-a690-c38b353cf763-node-exporter-textfile\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.674962 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.674800 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aff84e73-559f-4d6f-a690-c38b353cf763-root\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.674962 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.674827 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce140f37-9913-4f41-8522-377c123c7d65-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-mfg7x\" (UID: \"ce140f37-9913-4f41-8522-377c123c7d65\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mfg7x" Jul 23 13:08:51.674962 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.674855 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aff84e73-559f-4d6f-a690-c38b353cf763-node-exporter-tls\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.674962 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.674884 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aff84e73-559f-4d6f-a690-c38b353cf763-node-exporter-accelerators-collector-config\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.674962 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.674916 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce140f37-9913-4f41-8522-377c123c7d65-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-mfg7x\" (UID: \"ce140f37-9913-4f41-8522-377c123c7d65\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mfg7x" Jul 23 13:08:51.674962 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.674944 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce140f37-9913-4f41-8522-377c123c7d65-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-mfg7x\" (UID: \"ce140f37-9913-4f41-8522-377c123c7d65\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mfg7x" Jul 23 13:08:51.675323 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.674981 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aff84e73-559f-4d6f-a690-c38b353cf763-node-exporter-wtmp\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.675323 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.675026 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ce140f37-9913-4f41-8522-377c123c7d65-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-mfg7x\" (UID: \"ce140f37-9913-4f41-8522-377c123c7d65\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mfg7x" Jul 23 13:08:51.676185 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.675685 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ce140f37-9913-4f41-8522-377c123c7d65-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-mfg7x\" (UID: \"ce140f37-9913-4f41-8522-377c123c7d65\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mfg7x" Jul 23 13:08:51.676185 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.676149 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce140f37-9913-4f41-8522-377c123c7d65-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-mfg7x\" (UID: \"ce140f37-9913-4f41-8522-377c123c7d65\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mfg7x" Jul 23 13:08:51.678094 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.678073 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce140f37-9913-4f41-8522-377c123c7d65-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-mfg7x\" (UID: \"ce140f37-9913-4f41-8522-377c123c7d65\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mfg7x" Jul 23 13:08:51.682895 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.682866 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce140f37-9913-4f41-8522-377c123c7d65-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-mfg7x\" (UID: \"ce140f37-9913-4f41-8522-377c123c7d65\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mfg7x" Jul 23 13:08:51.684027 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.684004 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvxxz\" (UniqueName: \"kubernetes.io/projected/ce140f37-9913-4f41-8522-377c123c7d65-kube-api-access-xvxxz\") pod \"kube-state-metrics-56c5594674-mfg7x\" (UID: \"ce140f37-9913-4f41-8522-377c123c7d65\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mfg7x" Jul 23 13:08:51.775727 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.775699 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aff84e73-559f-4d6f-a690-c38b353cf763-node-exporter-wtmp\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.775930 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.775750 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aff84e73-559f-4d6f-a690-c38b353cf763-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.775930 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.775775 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aff84e73-559f-4d6f-a690-c38b353cf763-sys\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.775930 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.775813 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prl4j\" (UniqueName: \"kubernetes.io/projected/aff84e73-559f-4d6f-a690-c38b353cf763-kube-api-access-prl4j\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.775930 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.775842 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aff84e73-559f-4d6f-a690-c38b353cf763-metrics-client-ca\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.775930 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.775868 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aff84e73-559f-4d6f-a690-c38b353cf763-node-exporter-textfile\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.775930 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.775890 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aff84e73-559f-4d6f-a690-c38b353cf763-root\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.775930 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.775900 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aff84e73-559f-4d6f-a690-c38b353cf763-node-exporter-wtmp\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.775930 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.775917 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aff84e73-559f-4d6f-a690-c38b353cf763-node-exporter-tls\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.775930 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.775900 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aff84e73-559f-4d6f-a690-c38b353cf763-sys\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.776426 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.775955 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aff84e73-559f-4d6f-a690-c38b353cf763-node-exporter-accelerators-collector-config\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.776426 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.776207 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aff84e73-559f-4d6f-a690-c38b353cf763-root\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.776426 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.776308 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aff84e73-559f-4d6f-a690-c38b353cf763-node-exporter-textfile\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.776695 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.776612 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aff84e73-559f-4d6f-a690-c38b353cf763-metrics-client-ca\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.776695 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.776663 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aff84e73-559f-4d6f-a690-c38b353cf763-node-exporter-accelerators-collector-config\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.778420 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.778397 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aff84e73-559f-4d6f-a690-c38b353cf763-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.778540 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.778521 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aff84e73-559f-4d6f-a690-c38b353cf763-node-exporter-tls\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.784539 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.784518 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prl4j\" (UniqueName: \"kubernetes.io/projected/aff84e73-559f-4d6f-a690-c38b353cf763-kube-api-access-prl4j\") pod \"node-exporter-cbk6z\" (UID: \"aff84e73-559f-4d6f-a690-c38b353cf763\") " pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:51.870781 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.870748 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-mfg7x" Jul 23 13:08:51.875824 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:51.875798 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cbk6z" Jul 23 13:08:52.593080 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.591421 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 13:08:52.601455 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.601433 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.604525 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.604380 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 23 13:08:52.604525 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.604391 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 23 13:08:52.604525 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.604399 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 23 13:08:52.604525 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.604380 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-v7mzp\"" Jul 23 13:08:52.604525 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.604467 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 23 13:08:52.604772 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.604637 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 23 13:08:52.604772 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.604701 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 23 13:08:52.604869 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.604789 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 23 13:08:52.604869 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.604841 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 23 13:08:52.604981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.604967 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 23 13:08:52.613992 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.613968 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 13:08:52.682782 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.682745 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.682977 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.682799 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.682977 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.682869 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/73805294-484d-40d9-87ce-7b5ec6eaa3c2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.682977 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.682897 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73805294-484d-40d9-87ce-7b5ec6eaa3c2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.682977 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.682914 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mg8r\" (UniqueName: \"kubernetes.io/projected/73805294-484d-40d9-87ce-7b5ec6eaa3c2-kube-api-access-2mg8r\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.682977 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.682940 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/73805294-484d-40d9-87ce-7b5ec6eaa3c2-config-out\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.682977 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.682975 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/73805294-484d-40d9-87ce-7b5ec6eaa3c2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.683302 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.683001 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73805294-484d-40d9-87ce-7b5ec6eaa3c2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.683302 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.683030 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.683302 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.683061 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-web-config\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.683302 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.683122 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.683302 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.683144 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-config-volume\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.683302 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.683159 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.783815 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.783772 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/73805294-484d-40d9-87ce-7b5ec6eaa3c2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.783815 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.783820 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73805294-484d-40d9-87ce-7b5ec6eaa3c2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.784060 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.783852 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mg8r\" (UniqueName: \"kubernetes.io/projected/73805294-484d-40d9-87ce-7b5ec6eaa3c2-kube-api-access-2mg8r\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.784060 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.783875 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/73805294-484d-40d9-87ce-7b5ec6eaa3c2-config-out\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.784060 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.783899 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/73805294-484d-40d9-87ce-7b5ec6eaa3c2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.784060 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.783920 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73805294-484d-40d9-87ce-7b5ec6eaa3c2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.784060 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.783939 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.784060 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.783969 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-web-config\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.784060 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.784006 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.784441 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.784079 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-config-volume\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.784441 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.784094 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.784441 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.784129 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.784441 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.784163 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.784441 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:52.784284 2531 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 23 13:08:52.784441 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:08:52.784348 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-main-tls podName:73805294-484d-40d9-87ce-7b5ec6eaa3c2 nodeName:}" failed. No retries permitted until 2026-07-23 13:08:53.284327007 +0000 UTC m=+70.523603008 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "73805294-484d-40d9-87ce-7b5ec6eaa3c2") : secret "alertmanager-main-tls" not found Jul 23 13:08:52.784738 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.784676 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73805294-484d-40d9-87ce-7b5ec6eaa3c2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.784999 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.784978 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/73805294-484d-40d9-87ce-7b5ec6eaa3c2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.787283 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.786079 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73805294-484d-40d9-87ce-7b5ec6eaa3c2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.787283 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.787243 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/73805294-484d-40d9-87ce-7b5ec6eaa3c2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.787947 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.787917 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-web-config\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.788046 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.787923 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.788604 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.788582 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.788791 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.788725 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.789868 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.789848 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/73805294-484d-40d9-87ce-7b5ec6eaa3c2-config-out\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.790156 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.790133 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.790339 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.790282 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-config-volume\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:52.792816 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:52.792794 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mg8r\" (UniqueName: \"kubernetes.io/projected/73805294-484d-40d9-87ce-7b5ec6eaa3c2-kube-api-access-2mg8r\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:53.288490 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.288447 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:53.290859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.290834 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:53.510137 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.510097 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-8476f75c4-9vzd9"] Jul 23 13:08:53.514016 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.513991 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:08:53.547204 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.547180 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8476f75c4-9vzd9"] Jul 23 13:08:53.547325 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.547307 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.550171 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.550144 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 23 13:08:53.550761 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.550583 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 23 13:08:53.550761 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.550598 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-booe0rd6dss66\"" Jul 23 13:08:53.550761 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.550598 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 23 13:08:53.550761 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.550615 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-tx87w\"" Jul 23 13:08:53.550761 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.550659 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 23 13:08:53.550761 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.550610 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 23 13:08:53.556880 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:08:53.556798 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaff84e73_559f_4d6f_a690_c38b353cf763.slice/crio-100d9f938f002a090a9a22507d8ae7c8f0ca9a19f8e8cb0c1492c5787c4e333e WatchSource:0}: Error finding container 100d9f938f002a090a9a22507d8ae7c8f0ca9a19f8e8cb0c1492c5787c4e333e: Status 404 returned error can't find the container with id 100d9f938f002a090a9a22507d8ae7c8f0ca9a19f8e8cb0c1492c5787c4e333e Jul 23 13:08:53.590039 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.589843 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1992ea7e-ea92-4658-8598-3e5b316f69ff-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8476f75c4-9vzd9\" (UID: \"1992ea7e-ea92-4658-8598-3e5b316f69ff\") " pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.590039 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.589901 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pzlm\" (UniqueName: \"kubernetes.io/projected/1992ea7e-ea92-4658-8598-3e5b316f69ff-kube-api-access-8pzlm\") pod \"thanos-querier-8476f75c4-9vzd9\" (UID: \"1992ea7e-ea92-4658-8598-3e5b316f69ff\") " pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.590039 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.589972 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1992ea7e-ea92-4658-8598-3e5b316f69ff-metrics-client-ca\") pod \"thanos-querier-8476f75c4-9vzd9\" (UID: \"1992ea7e-ea92-4658-8598-3e5b316f69ff\") " pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.590039 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.590021 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1992ea7e-ea92-4658-8598-3e5b316f69ff-secret-grpc-tls\") pod \"thanos-querier-8476f75c4-9vzd9\" (UID: \"1992ea7e-ea92-4658-8598-3e5b316f69ff\") " pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.590261 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.590053 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1992ea7e-ea92-4658-8598-3e5b316f69ff-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8476f75c4-9vzd9\" (UID: \"1992ea7e-ea92-4658-8598-3e5b316f69ff\") " pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.590261 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.590123 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1992ea7e-ea92-4658-8598-3e5b316f69ff-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8476f75c4-9vzd9\" (UID: \"1992ea7e-ea92-4658-8598-3e5b316f69ff\") " pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.590261 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.590154 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1992ea7e-ea92-4658-8598-3e5b316f69ff-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8476f75c4-9vzd9\" (UID: \"1992ea7e-ea92-4658-8598-3e5b316f69ff\") " pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.590261 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.590183 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1992ea7e-ea92-4658-8598-3e5b316f69ff-secret-thanos-querier-tls\") pod \"thanos-querier-8476f75c4-9vzd9\" (UID: \"1992ea7e-ea92-4658-8598-3e5b316f69ff\") " pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.690796 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.690764 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pzlm\" (UniqueName: \"kubernetes.io/projected/1992ea7e-ea92-4658-8598-3e5b316f69ff-kube-api-access-8pzlm\") pod \"thanos-querier-8476f75c4-9vzd9\" (UID: \"1992ea7e-ea92-4658-8598-3e5b316f69ff\") " pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.691164 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.690819 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1992ea7e-ea92-4658-8598-3e5b316f69ff-metrics-client-ca\") pod \"thanos-querier-8476f75c4-9vzd9\" (UID: \"1992ea7e-ea92-4658-8598-3e5b316f69ff\") " pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.691164 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.690858 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1992ea7e-ea92-4658-8598-3e5b316f69ff-secret-grpc-tls\") pod \"thanos-querier-8476f75c4-9vzd9\" (UID: \"1992ea7e-ea92-4658-8598-3e5b316f69ff\") " pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.691164 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.690889 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1992ea7e-ea92-4658-8598-3e5b316f69ff-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8476f75c4-9vzd9\" (UID: \"1992ea7e-ea92-4658-8598-3e5b316f69ff\") " pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.691164 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.690942 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1992ea7e-ea92-4658-8598-3e5b316f69ff-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8476f75c4-9vzd9\" (UID: \"1992ea7e-ea92-4658-8598-3e5b316f69ff\") " pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.691164 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.690969 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1992ea7e-ea92-4658-8598-3e5b316f69ff-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8476f75c4-9vzd9\" (UID: \"1992ea7e-ea92-4658-8598-3e5b316f69ff\") " pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.691164 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.690997 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1992ea7e-ea92-4658-8598-3e5b316f69ff-secret-thanos-querier-tls\") pod \"thanos-querier-8476f75c4-9vzd9\" (UID: \"1992ea7e-ea92-4658-8598-3e5b316f69ff\") " pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.691164 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.691025 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1992ea7e-ea92-4658-8598-3e5b316f69ff-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8476f75c4-9vzd9\" (UID: \"1992ea7e-ea92-4658-8598-3e5b316f69ff\") " pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.692980 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.692946 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1992ea7e-ea92-4658-8598-3e5b316f69ff-metrics-client-ca\") pod \"thanos-querier-8476f75c4-9vzd9\" (UID: \"1992ea7e-ea92-4658-8598-3e5b316f69ff\") " pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.694788 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.694743 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1992ea7e-ea92-4658-8598-3e5b316f69ff-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8476f75c4-9vzd9\" (UID: \"1992ea7e-ea92-4658-8598-3e5b316f69ff\") " pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.695290 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.695250 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1992ea7e-ea92-4658-8598-3e5b316f69ff-secret-grpc-tls\") pod \"thanos-querier-8476f75c4-9vzd9\" (UID: \"1992ea7e-ea92-4658-8598-3e5b316f69ff\") " pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.696649 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.696623 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-mfg7x"] Jul 23 13:08:53.696909 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.696884 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1992ea7e-ea92-4658-8598-3e5b316f69ff-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8476f75c4-9vzd9\" (UID: \"1992ea7e-ea92-4658-8598-3e5b316f69ff\") " pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.697817 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.697797 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1992ea7e-ea92-4658-8598-3e5b316f69ff-secret-thanos-querier-tls\") pod \"thanos-querier-8476f75c4-9vzd9\" (UID: \"1992ea7e-ea92-4658-8598-3e5b316f69ff\") " pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.698970 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.698947 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1992ea7e-ea92-4658-8598-3e5b316f69ff-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8476f75c4-9vzd9\" (UID: \"1992ea7e-ea92-4658-8598-3e5b316f69ff\") " pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.699192 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.699170 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1992ea7e-ea92-4658-8598-3e5b316f69ff-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8476f75c4-9vzd9\" (UID: \"1992ea7e-ea92-4658-8598-3e5b316f69ff\") " pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.699789 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.699772 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pzlm\" (UniqueName: \"kubernetes.io/projected/1992ea7e-ea92-4658-8598-3e5b316f69ff-kube-api-access-8pzlm\") pod \"thanos-querier-8476f75c4-9vzd9\" (UID: \"1992ea7e-ea92-4658-8598-3e5b316f69ff\") " pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.705380 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:08:53.705336 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce140f37_9913_4f41_8522_377c123c7d65.slice/crio-fd28a36274ae227431ab5b6f86c08615dee8371a13558adb7a75140819b784e7 WatchSource:0}: Error finding container fd28a36274ae227431ab5b6f86c08615dee8371a13558adb7a75140819b784e7: Status 404 returned error can't find the container with id fd28a36274ae227431ab5b6f86c08615dee8371a13558adb7a75140819b784e7 Jul 23 13:08:53.718471 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.718450 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 13:08:53.720459 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:08:53.720439 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73805294_484d_40d9_87ce_7b5ec6eaa3c2.slice/crio-55da06a65b7fff8ab71e03a3c20f54ac13e8f5e3213f8e8405e88e7a24f72c17 WatchSource:0}: Error finding container 55da06a65b7fff8ab71e03a3c20f54ac13e8f5e3213f8e8405e88e7a24f72c17: Status 404 returned error can't find the container with id 55da06a65b7fff8ab71e03a3c20f54ac13e8f5e3213f8e8405e88e7a24f72c17 Jul 23 13:08:53.847784 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.847740 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tc68c" event={"ID":"65c6347a-d5cc-4964-b5dd-b5b4098e0c5e","Type":"ContainerStarted","Data":"a311bb951831a3a185d1f3749e3957686198e64a90039c7fa44246ac9c931125"} Jul 23 13:08:53.847970 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.847845 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:08:53.849303 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.849279 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vfgv8" event={"ID":"5df5e8b8-2bc3-4788-bf77-4f32a3d52f65","Type":"ContainerStarted","Data":"bed4ea5b6862b70156f0dac93f342cccea18a2c1e284b420bc884bac443ea2dd"} Jul 23 13:08:53.850592 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.850570 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73805294-484d-40d9-87ce-7b5ec6eaa3c2","Type":"ContainerStarted","Data":"55da06a65b7fff8ab71e03a3c20f54ac13e8f5e3213f8e8405e88e7a24f72c17"} Jul 23 13:08:53.851798 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.851780 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-mfg7x" event={"ID":"ce140f37-9913-4f41-8522-377c123c7d65","Type":"ContainerStarted","Data":"fd28a36274ae227431ab5b6f86c08615dee8371a13558adb7a75140819b784e7"} Jul 23 13:08:53.852907 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.852889 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cbk6z" event={"ID":"aff84e73-559f-4d6f-a690-c38b353cf763","Type":"ContainerStarted","Data":"100d9f938f002a090a9a22507d8ae7c8f0ca9a19f8e8cb0c1492c5787c4e333e"} Jul 23 13:08:53.863013 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.862974 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-tc68c" podStartSLOduration=64.89095118 podStartE2EDuration="1m9.862960795s" totalImagesPullingTime="4.972009615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:07:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:08:53.86196672 +0000 UTC m=+71.101242696" watchObservedRunningTime="2026-07-23 13:08:53.862960795 +0000 UTC m=+71.102236771" Jul 23 13:08:53.867253 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.867231 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:08:53.880674 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:53.880640 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vfgv8" podStartSLOduration=64.593254094 podStartE2EDuration="1m9.880627057s" totalImagesPullingTime="5.287372963s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:07:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:08:53.880282564 +0000 UTC m=+71.119558540" watchObservedRunningTime="2026-07-23 13:08:53.880627057 +0000 UTC m=+71.119903010" Jul 23 13:08:54.018898 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:54.018875 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8476f75c4-9vzd9"] Jul 23 13:08:54.021133 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:08:54.021100 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1992ea7e_ea92_4658_8598_3e5b316f69ff.slice/crio-5b3c20a6031ccc08632f88b76c9d1967692f37bc23907e8c78360acf427d4d3e WatchSource:0}: Error finding container 5b3c20a6031ccc08632f88b76c9d1967692f37bc23907e8c78360acf427d4d3e: Status 404 returned error can't find the container with id 5b3c20a6031ccc08632f88b76c9d1967692f37bc23907e8c78360acf427d4d3e Jul 23 13:08:54.859105 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:54.858596 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" event={"ID":"1992ea7e-ea92-4658-8598-3e5b316f69ff","Type":"ContainerStarted","Data":"5b3c20a6031ccc08632f88b76c9d1967692f37bc23907e8c78360acf427d4d3e"} Jul 23 13:08:55.863027 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:55.862996 2531 generic.go:356] "Generic (PLEG): container finished" podID="aff84e73-559f-4d6f-a690-c38b353cf763" containerID="bc9db7fa548de417f3d842b4973eebe94ac264ec8a2823ee050118be514afe0b" exitCode=0 Jul 23 13:08:55.863494 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:55.863092 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cbk6z" event={"ID":"aff84e73-559f-4d6f-a690-c38b353cf763","Type":"ContainerDied","Data":"bc9db7fa548de417f3d842b4973eebe94ac264ec8a2823ee050118be514afe0b"} Jul 23 13:08:55.864748 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:55.864729 2531 generic.go:356] "Generic (PLEG): container finished" podID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerID="709bc0ed85e5eeae1a25108354922726bb0bf898cd0fc41baef514e0fc22d401" exitCode=0 Jul 23 13:08:55.864872 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:55.864790 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73805294-484d-40d9-87ce-7b5ec6eaa3c2","Type":"ContainerDied","Data":"709bc0ed85e5eeae1a25108354922726bb0bf898cd0fc41baef514e0fc22d401"} Jul 23 13:08:55.867133 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:55.867104 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-mfg7x" event={"ID":"ce140f37-9913-4f41-8522-377c123c7d65","Type":"ContainerStarted","Data":"3e039fde6d9357cdecfc5d81e9a554fcef6e3e7d4533150297f7227bf37531bd"} Jul 23 13:08:55.867260 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:55.867156 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-mfg7x" event={"ID":"ce140f37-9913-4f41-8522-377c123c7d65","Type":"ContainerStarted","Data":"501655fd13269410c3a55ed9e2d0ac91870e6d25276aa8fbf0fa4f480ea8c650"} Jul 23 13:08:55.867260 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:55.867186 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-mfg7x" event={"ID":"ce140f37-9913-4f41-8522-377c123c7d65","Type":"ContainerStarted","Data":"12e1d8b47fc6e1d80b35a5b6165d2eb943951ec8c8e5aedda9f2567a51128cd5"} Jul 23 13:08:55.934813 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:55.934763 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-mfg7x" podStartSLOduration=3.190550791 podStartE2EDuration="4.934747087s" totalImagesPullingTime="1.744196296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:08:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:08:55.933171481 +0000 UTC m=+73.172447457" watchObservedRunningTime="2026-07-23 13:08:55.934747087 +0000 UTC m=+73.174023064" Jul 23 13:08:56.311802 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.311764 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-fq7mk"] Jul 23 13:08:56.314316 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.314287 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-fq7mk" Jul 23 13:08:56.316861 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.316838 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-4d4cs\"" Jul 23 13:08:56.317294 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.317276 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 23 13:08:56.327500 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.327475 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-fq7mk"] Jul 23 13:08:56.411675 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.411641 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0d5e4e55-0fd8-4802-bd5a-a970f39b27ab-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-fq7mk\" (UID: \"0d5e4e55-0fd8-4802-bd5a-a970f39b27ab\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-fq7mk" Jul 23 13:08:56.411833 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.411702 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gtfr\" (UniqueName: \"kubernetes.io/projected/0d5e4e55-0fd8-4802-bd5a-a970f39b27ab-kube-api-access-2gtfr\") pod \"monitoring-plugin-7b7b7bc8bf-fq7mk\" (UID: \"0d5e4e55-0fd8-4802-bd5a-a970f39b27ab\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-fq7mk" Jul 23 13:08:56.512561 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.512527 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0d5e4e55-0fd8-4802-bd5a-a970f39b27ab-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-fq7mk\" (UID: \"0d5e4e55-0fd8-4802-bd5a-a970f39b27ab\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-fq7mk" Jul 23 13:08:56.512728 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.512586 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gtfr\" (UniqueName: \"kubernetes.io/projected/0d5e4e55-0fd8-4802-bd5a-a970f39b27ab-kube-api-access-2gtfr\") pod \"monitoring-plugin-7b7b7bc8bf-fq7mk\" (UID: \"0d5e4e55-0fd8-4802-bd5a-a970f39b27ab\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-fq7mk" Jul 23 13:08:56.515667 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.515636 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0d5e4e55-0fd8-4802-bd5a-a970f39b27ab-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-fq7mk\" (UID: \"0d5e4e55-0fd8-4802-bd5a-a970f39b27ab\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-fq7mk" Jul 23 13:08:56.522215 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.522183 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gtfr\" (UniqueName: \"kubernetes.io/projected/0d5e4e55-0fd8-4802-bd5a-a970f39b27ab-kube-api-access-2gtfr\") pod \"monitoring-plugin-7b7b7bc8bf-fq7mk\" (UID: \"0d5e4e55-0fd8-4802-bd5a-a970f39b27ab\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-fq7mk" Jul 23 13:08:56.624145 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.624064 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-fq7mk" Jul 23 13:08:56.731092 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.731060 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5676dcddfb-8zklg"] Jul 23 13:08:56.735394 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.734967 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:08:56.736002 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.735980 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5676dcddfb-8zklg"] Jul 23 13:08:56.738282 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.738257 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mqvzm\"" Jul 23 13:08:56.738453 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.738433 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 23 13:08:56.738599 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.738203 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 23 13:08:56.738880 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.738863 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 23 13:08:56.739424 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.739408 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 23 13:08:56.739825 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.739807 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 23 13:08:56.739878 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.739867 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 23 13:08:56.739955 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.739933 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 23 13:08:56.745453 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.745431 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 23 13:08:56.778515 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.778486 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-fq7mk"] Jul 23 13:08:56.782535 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:08:56.782497 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d5e4e55_0fd8_4802_bd5a_a970f39b27ab.slice/crio-14783d2903dc7184c608d0d312ad773ab0a7dde911292bffa8a1ab6818ea63de WatchSource:0}: Error finding container 14783d2903dc7184c608d0d312ad773ab0a7dde911292bffa8a1ab6818ea63de: Status 404 returned error can't find the container with id 14783d2903dc7184c608d0d312ad773ab0a7dde911292bffa8a1ab6818ea63de Jul 23 13:08:56.814773 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.814744 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3828321a-3058-4e5f-a076-3689380b9a9a-service-ca\") pod \"console-5676dcddfb-8zklg\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:08:56.814889 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.814779 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3828321a-3058-4e5f-a076-3689380b9a9a-console-serving-cert\") pod \"console-5676dcddfb-8zklg\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:08:56.814889 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.814802 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3828321a-3058-4e5f-a076-3689380b9a9a-trusted-ca-bundle\") pod \"console-5676dcddfb-8zklg\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:08:56.814889 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.814852 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p9q8\" (UniqueName: \"kubernetes.io/projected/3828321a-3058-4e5f-a076-3689380b9a9a-kube-api-access-6p9q8\") pod \"console-5676dcddfb-8zklg\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:08:56.814889 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.814885 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3828321a-3058-4e5f-a076-3689380b9a9a-console-config\") pod \"console-5676dcddfb-8zklg\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:08:56.815035 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.814933 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3828321a-3058-4e5f-a076-3689380b9a9a-console-oauth-config\") pod \"console-5676dcddfb-8zklg\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:08:56.815035 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.814971 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3828321a-3058-4e5f-a076-3689380b9a9a-oauth-serving-cert\") pod \"console-5676dcddfb-8zklg\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:08:56.872111 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.872066 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-fq7mk" event={"ID":"0d5e4e55-0fd8-4802-bd5a-a970f39b27ab","Type":"ContainerStarted","Data":"14783d2903dc7184c608d0d312ad773ab0a7dde911292bffa8a1ab6818ea63de"} Jul 23 13:08:56.874368 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.874289 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cbk6z" event={"ID":"aff84e73-559f-4d6f-a690-c38b353cf763","Type":"ContainerStarted","Data":"6c9eaa56f88a39580935185b0fc945ed9a80b2ea357e9dbdf67b6c59131b748e"} Jul 23 13:08:56.874368 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.874341 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cbk6z" event={"ID":"aff84e73-559f-4d6f-a690-c38b353cf763","Type":"ContainerStarted","Data":"91539f8012222bfa5a3b3f8f8af602ed69d1cf430cac5f62a512880dfec3a239"} Jul 23 13:08:56.891939 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.891784 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cbk6z" podStartSLOduration=3.912620043 podStartE2EDuration="5.891765197s" totalImagesPullingTime="1.889788281s" totalInitContainerRuntime="89.356873ms" isStatefulPod=true podCreationTimestamp="2026-07-23 13:08:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:08:56.8914378 +0000 UTC m=+74.130713778" watchObservedRunningTime="2026-07-23 13:08:56.891765197 +0000 UTC m=+74.131041174" Jul 23 13:08:56.916138 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.916112 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3828321a-3058-4e5f-a076-3689380b9a9a-oauth-serving-cert\") pod \"console-5676dcddfb-8zklg\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:08:56.916267 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.916166 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3828321a-3058-4e5f-a076-3689380b9a9a-service-ca\") pod \"console-5676dcddfb-8zklg\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:08:56.916267 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.916197 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3828321a-3058-4e5f-a076-3689380b9a9a-console-serving-cert\") pod \"console-5676dcddfb-8zklg\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:08:56.916267 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.916229 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3828321a-3058-4e5f-a076-3689380b9a9a-trusted-ca-bundle\") pod \"console-5676dcddfb-8zklg\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:08:56.916483 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.916462 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p9q8\" (UniqueName: \"kubernetes.io/projected/3828321a-3058-4e5f-a076-3689380b9a9a-kube-api-access-6p9q8\") pod \"console-5676dcddfb-8zklg\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:08:56.916845 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.916821 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3828321a-3058-4e5f-a076-3689380b9a9a-console-config\") pod \"console-5676dcddfb-8zklg\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:08:56.917117 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.916864 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3828321a-3058-4e5f-a076-3689380b9a9a-console-oauth-config\") pod \"console-5676dcddfb-8zklg\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:08:56.917117 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.916952 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3828321a-3058-4e5f-a076-3689380b9a9a-oauth-serving-cert\") pod \"console-5676dcddfb-8zklg\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:08:56.917117 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.917027 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3828321a-3058-4e5f-a076-3689380b9a9a-service-ca\") pod \"console-5676dcddfb-8zklg\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:08:56.917403 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.917348 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3828321a-3058-4e5f-a076-3689380b9a9a-console-config\") pod \"console-5676dcddfb-8zklg\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:08:56.917723 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.917703 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3828321a-3058-4e5f-a076-3689380b9a9a-trusted-ca-bundle\") pod \"console-5676dcddfb-8zklg\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:08:56.918892 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.918871 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3828321a-3058-4e5f-a076-3689380b9a9a-console-serving-cert\") pod \"console-5676dcddfb-8zklg\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:08:56.919130 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.919114 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3828321a-3058-4e5f-a076-3689380b9a9a-console-oauth-config\") pod \"console-5676dcddfb-8zklg\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:08:56.927098 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:56.927081 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p9q8\" (UniqueName: \"kubernetes.io/projected/3828321a-3058-4e5f-a076-3689380b9a9a-kube-api-access-6p9q8\") pod \"console-5676dcddfb-8zklg\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:08:57.049570 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:57.049536 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:08:57.219698 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:57.219666 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5676dcddfb-8zklg"] Jul 23 13:08:57.736981 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:08:57.736942 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3828321a_3058_4e5f_a076_3689380b9a9a.slice/crio-8ef27ee361cb2c0c239e5738178fd34cf3067e0e0bc70200639bf44bd3556248 WatchSource:0}: Error finding container 8ef27ee361cb2c0c239e5738178fd34cf3067e0e0bc70200639bf44bd3556248: Status 404 returned error can't find the container with id 8ef27ee361cb2c0c239e5738178fd34cf3067e0e0bc70200639bf44bd3556248 Jul 23 13:08:57.879995 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:57.879883 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5676dcddfb-8zklg" event={"ID":"3828321a-3058-4e5f-a076-3689380b9a9a","Type":"ContainerStarted","Data":"8ef27ee361cb2c0c239e5738178fd34cf3067e0e0bc70200639bf44bd3556248"} Jul 23 13:08:58.892499 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:58.892255 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73805294-484d-40d9-87ce-7b5ec6eaa3c2","Type":"ContainerStarted","Data":"e04cb036475a8027f4724984710af645bcafd63f96155078f053876f9fb42262"} Jul 23 13:08:58.892499 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:58.892399 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73805294-484d-40d9-87ce-7b5ec6eaa3c2","Type":"ContainerStarted","Data":"39c51f6bdf24f3153d90875a23be3eca104679cb3df0fb9dc67c6218685a45e8"} Jul 23 13:08:58.892499 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:58.892443 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73805294-484d-40d9-87ce-7b5ec6eaa3c2","Type":"ContainerStarted","Data":"4d71534fc988a00bb0a8d73ad5479e7bca610255ed7347cfd0958d4074fe1f51"} Jul 23 13:08:58.892499 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:58.892468 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73805294-484d-40d9-87ce-7b5ec6eaa3c2","Type":"ContainerStarted","Data":"aa545e9d5b0b24c6676a6ed9b1df1c12e198f9b3ff1022fbe17927e0bb4d37c9"} Jul 23 13:08:58.894383 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:58.894316 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-fq7mk" event={"ID":"0d5e4e55-0fd8-4802-bd5a-a970f39b27ab","Type":"ContainerStarted","Data":"d4dafbc25d7adc847d3353b8890bc1e4381199f00a10484c3c5f46a74ee70d39"} Jul 23 13:08:58.894676 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:58.894625 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-fq7mk" Jul 23 13:08:58.896909 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:58.896882 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" event={"ID":"1992ea7e-ea92-4658-8598-3e5b316f69ff","Type":"ContainerStarted","Data":"5be2b2ca9f5ae9d199b4c2cc11fdc9aa4583cc4a8868e0cd595ae3bfb3d6d56d"} Jul 23 13:08:58.897025 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:58.896953 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" event={"ID":"1992ea7e-ea92-4658-8598-3e5b316f69ff","Type":"ContainerStarted","Data":"e00084bcba73582af1de6c07d1ecc6b9655f2b2fbe5ec86504faa9e56cbf935c"} Jul 23 13:08:58.897025 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:58.896995 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" event={"ID":"1992ea7e-ea92-4658-8598-3e5b316f69ff","Type":"ContainerStarted","Data":"6cca81398dc8a7718b5fa2db7bf630ae8c75a766ea758aac23eccac87d687527"} Jul 23 13:08:58.900768 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:58.900747 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-fq7mk" Jul 23 13:08:58.910525 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:58.910472 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-fq7mk" podStartSLOduration=1.218037316 podStartE2EDuration="2.910453782s" totalImagesPullingTime="1.692416466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:08:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:08:58.909613653 +0000 UTC m=+76.148889625" watchObservedRunningTime="2026-07-23 13:08:58.910453782 +0000 UTC m=+76.149729758" Jul 23 13:08:59.904043 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:08:59.904007 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73805294-484d-40d9-87ce-7b5ec6eaa3c2","Type":"ContainerStarted","Data":"568d77d5e067b753efb61217378d735e64238f70026787e1426407da9236d023"} Jul 23 13:09:00.910367 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:00.910277 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5676dcddfb-8zklg" event={"ID":"3828321a-3058-4e5f-a076-3689380b9a9a","Type":"ContainerStarted","Data":"f84bf8648197257d9438e29ffa30d8b49f404c93b7eece7546244e19f9ddaf48"} Jul 23 13:09:00.914418 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:00.914384 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" event={"ID":"1992ea7e-ea92-4658-8598-3e5b316f69ff","Type":"ContainerStarted","Data":"1b494e0dbaf3543e52914c931530df755df4c2502513e9150cbb7ec0a645c200"} Jul 23 13:09:00.914518 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:00.914473 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" event={"ID":"1992ea7e-ea92-4658-8598-3e5b316f69ff","Type":"ContainerStarted","Data":"e16dbe9900c71b2aaec41d2db43d1c04ae73b3fd7e4a15644dbc325ceeb18c1e"} Jul 23 13:09:00.928311 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:00.928269 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5676dcddfb-8zklg" podStartSLOduration=1.9598750969999998 podStartE2EDuration="4.928255007s" totalImagesPullingTime="2.96837991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:08:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:09:00.927554705 +0000 UTC m=+78.166830681" watchObservedRunningTime="2026-07-23 13:09:00.928255007 +0000 UTC m=+78.167530980" Jul 23 13:09:00.950061 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:00.950015 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" podStartSLOduration=1.540040297 podStartE2EDuration="7.949996494s" totalImagesPullingTime="6.409956197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:08:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:09:00.948706549 +0000 UTC m=+78.187982526" watchObservedRunningTime="2026-07-23 13:09:00.949996494 +0000 UTC m=+78.189272521" Jul 23 13:09:01.920818 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:01.920774 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" event={"ID":"1992ea7e-ea92-4658-8598-3e5b316f69ff","Type":"ContainerStarted","Data":"a77d6f47603f35a843e4a938a87b9287c7a21b1f07eee84c90c23c849ebec0ed"} Jul 23 13:09:01.922101 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:01.922077 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:09:01.928416 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:01.928388 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73805294-484d-40d9-87ce-7b5ec6eaa3c2","Type":"ContainerStarted","Data":"0c507f18ba55ff4a23504d9894a4be93cefa734e6d16af6792a67c7875d8e550"} Jul 23 13:09:01.956708 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:01.956657 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.605193163 podStartE2EDuration="9.956636835s" totalImagesPullingTime="6.337155027s" totalInitContainerRuntime="14.288645ms" isStatefulPod=false podCreationTimestamp="2026-07-23 13:08:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:09:01.955183244 +0000 UTC m=+79.194459245" watchObservedRunningTime="2026-07-23 13:09:01.956636835 +0000 UTC m=+79.195912812" Jul 23 13:09:07.050507 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:07.050392 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:09:07.050507 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:07.050452 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:09:07.054911 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:07.054890 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:09:07.938501 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:07.938475 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-8476f75c4-9vzd9" Jul 23 13:09:07.957555 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:07.957532 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:09:09.295802 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:09.295767 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_73805294-484d-40d9-87ce-7b5ec6eaa3c2/init-config-reloader/0.log" Jul 23 13:09:09.496060 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:09.496032 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_73805294-484d-40d9-87ce-7b5ec6eaa3c2/alertmanager/0.log" Jul 23 13:09:09.696044 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:09.696017 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_73805294-484d-40d9-87ce-7b5ec6eaa3c2/config-reloader/0.log" Jul 23 13:09:09.904155 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:09.904128 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_73805294-484d-40d9-87ce-7b5ec6eaa3c2/kube-rbac-proxy-web/0.log" Jul 23 13:09:10.096218 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:10.096180 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_73805294-484d-40d9-87ce-7b5ec6eaa3c2/kube-rbac-proxy/0.log" Jul 23 13:09:10.296122 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:10.296095 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_73805294-484d-40d9-87ce-7b5ec6eaa3c2/kube-rbac-proxy-metric/0.log" Jul 23 13:09:10.498272 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:10.498172 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_73805294-484d-40d9-87ce-7b5ec6eaa3c2/prom-label-proxy/0.log" Jul 23 13:09:10.896589 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:10.896562 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-mfg7x_ce140f37-9913-4f41-8522-377c123c7d65/kube-state-metrics/0.log" Jul 23 13:09:11.095590 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:11.095561 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-mfg7x_ce140f37-9913-4f41-8522-377c123c7d65/kube-rbac-proxy-main/0.log" Jul 23 13:09:11.296530 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:11.296453 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-mfg7x_ce140f37-9913-4f41-8522-377c123c7d65/kube-rbac-proxy-self/0.log" Jul 23 13:09:11.695235 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:11.695205 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-fq7mk_0d5e4e55-0fd8-4802-bd5a-a970f39b27ab/monitoring-plugin/0.log" Jul 23 13:09:11.895481 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:11.895457 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cbk6z_aff84e73-559f-4d6f-a690-c38b353cf763/init-textfile/0.log" Jul 23 13:09:12.096963 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:12.096934 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cbk6z_aff84e73-559f-4d6f-a690-c38b353cf763/node-exporter/0.log" Jul 23 13:09:12.296755 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:12.296729 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cbk6z_aff84e73-559f-4d6f-a690-c38b353cf763/kube-rbac-proxy/0.log" Jul 23 13:09:16.296205 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:16.296174 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8476f75c4-9vzd9_1992ea7e-ea92-4658-8598-3e5b316f69ff/thanos-query/0.log" Jul 23 13:09:16.495790 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:16.495746 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8476f75c4-9vzd9_1992ea7e-ea92-4658-8598-3e5b316f69ff/kube-rbac-proxy-web/0.log" Jul 23 13:09:16.695594 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:16.695567 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8476f75c4-9vzd9_1992ea7e-ea92-4658-8598-3e5b316f69ff/kube-rbac-proxy/0.log" Jul 23 13:09:16.896035 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:16.896004 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8476f75c4-9vzd9_1992ea7e-ea92-4658-8598-3e5b316f69ff/prom-label-proxy/0.log" Jul 23 13:09:17.097069 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:17.097038 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8476f75c4-9vzd9_1992ea7e-ea92-4658-8598-3e5b316f69ff/kube-rbac-proxy-rules/0.log" Jul 23 13:09:17.297000 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:17.296971 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8476f75c4-9vzd9_1992ea7e-ea92-4658-8598-3e5b316f69ff/kube-rbac-proxy-metrics/0.log" Jul 23 13:09:18.096149 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:18.096120 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5676dcddfb-8zklg_3828321a-3058-4e5f-a076-3689380b9a9a/console/0.log" Jul 23 13:09:19.295622 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:19.295595 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cd2jt_8261ecc7-323a-4dd7-8a6a-43381de92427/serve-healthcheck-canary/0.log" Jul 23 13:09:24.861282 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:09:24.861249 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-tc68c" Jul 23 13:10:11.819576 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:11.819539 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 13:10:11.820071 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:11.819939 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="alertmanager" containerID="cri-o://aa545e9d5b0b24c6676a6ed9b1df1c12e198f9b3ff1022fbe17927e0bb4d37c9" gracePeriod=120 Jul 23 13:10:11.820071 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:11.819988 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="kube-rbac-proxy-metric" containerID="cri-o://568d77d5e067b753efb61217378d735e64238f70026787e1426407da9236d023" gracePeriod=120 Jul 23 13:10:11.820071 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:11.820038 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="prom-label-proxy" containerID="cri-o://0c507f18ba55ff4a23504d9894a4be93cefa734e6d16af6792a67c7875d8e550" gracePeriod=120 Jul 23 13:10:11.820071 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:11.820066 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="config-reloader" containerID="cri-o://4d71534fc988a00bb0a8d73ad5479e7bca610255ed7347cfd0958d4074fe1f51" gracePeriod=120 Jul 23 13:10:11.820269 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:11.820062 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="kube-rbac-proxy" containerID="cri-o://e04cb036475a8027f4724984710af645bcafd63f96155078f053876f9fb42262" gracePeriod=120 Jul 23 13:10:11.820269 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:11.820064 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="kube-rbac-proxy-web" containerID="cri-o://39c51f6bdf24f3153d90875a23be3eca104679cb3df0fb9dc67c6218685a45e8" gracePeriod=120 Jul 23 13:10:12.190166 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:12.190080 2531 generic.go:356] "Generic (PLEG): container finished" podID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerID="0c507f18ba55ff4a23504d9894a4be93cefa734e6d16af6792a67c7875d8e550" exitCode=0 Jul 23 13:10:12.190166 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:12.190106 2531 generic.go:356] "Generic (PLEG): container finished" podID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerID="e04cb036475a8027f4724984710af645bcafd63f96155078f053876f9fb42262" exitCode=0 Jul 23 13:10:12.190166 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:12.190113 2531 generic.go:356] "Generic (PLEG): container finished" podID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerID="4d71534fc988a00bb0a8d73ad5479e7bca610255ed7347cfd0958d4074fe1f51" exitCode=0 Jul 23 13:10:12.190166 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:12.190118 2531 generic.go:356] "Generic (PLEG): container finished" podID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerID="aa545e9d5b0b24c6676a6ed9b1df1c12e198f9b3ff1022fbe17927e0bb4d37c9" exitCode=0 Jul 23 13:10:12.190410 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:12.190156 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73805294-484d-40d9-87ce-7b5ec6eaa3c2","Type":"ContainerDied","Data":"0c507f18ba55ff4a23504d9894a4be93cefa734e6d16af6792a67c7875d8e550"} Jul 23 13:10:12.190410 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:12.190227 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73805294-484d-40d9-87ce-7b5ec6eaa3c2","Type":"ContainerDied","Data":"e04cb036475a8027f4724984710af645bcafd63f96155078f053876f9fb42262"} Jul 23 13:10:12.190410 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:12.190253 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73805294-484d-40d9-87ce-7b5ec6eaa3c2","Type":"ContainerDied","Data":"4d71534fc988a00bb0a8d73ad5479e7bca610255ed7347cfd0958d4074fe1f51"} Jul 23 13:10:12.190410 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:12.190279 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73805294-484d-40d9-87ce-7b5ec6eaa3c2","Type":"ContainerDied","Data":"aa545e9d5b0b24c6676a6ed9b1df1c12e198f9b3ff1022fbe17927e0bb4d37c9"} Jul 23 13:10:13.058755 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.058731 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.108816 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.108786 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-cluster-tls-config\") pod \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " Jul 23 13:10:13.108990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.108834 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73805294-484d-40d9-87ce-7b5ec6eaa3c2-metrics-client-ca\") pod \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " Jul 23 13:10:13.108990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.108851 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-web-config\") pod \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " Jul 23 13:10:13.108990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.108870 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " Jul 23 13:10:13.108990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.108893 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/73805294-484d-40d9-87ce-7b5ec6eaa3c2-tls-assets\") pod \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " Jul 23 13:10:13.108990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.108915 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mg8r\" (UniqueName: \"kubernetes.io/projected/73805294-484d-40d9-87ce-7b5ec6eaa3c2-kube-api-access-2mg8r\") pod \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " Jul 23 13:10:13.108990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.108939 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-config-volume\") pod \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " Jul 23 13:10:13.108990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.108977 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73805294-484d-40d9-87ce-7b5ec6eaa3c2-alertmanager-trusted-ca-bundle\") pod \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " Jul 23 13:10:13.109369 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.109012 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/73805294-484d-40d9-87ce-7b5ec6eaa3c2-alertmanager-main-db\") pod \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " Jul 23 13:10:13.109369 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.109053 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/73805294-484d-40d9-87ce-7b5ec6eaa3c2-config-out\") pod \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " Jul 23 13:10:13.109369 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.109086 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-kube-rbac-proxy-web\") pod \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " Jul 23 13:10:13.109369 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.109136 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-kube-rbac-proxy\") pod \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " Jul 23 13:10:13.109369 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.109168 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-main-tls\") pod \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\" (UID: \"73805294-484d-40d9-87ce-7b5ec6eaa3c2\") " Jul 23 13:10:13.109672 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.109403 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73805294-484d-40d9-87ce-7b5ec6eaa3c2-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "73805294-484d-40d9-87ce-7b5ec6eaa3c2" (UID: "73805294-484d-40d9-87ce-7b5ec6eaa3c2"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:10:13.110188 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.109853 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73805294-484d-40d9-87ce-7b5ec6eaa3c2-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "73805294-484d-40d9-87ce-7b5ec6eaa3c2" (UID: "73805294-484d-40d9-87ce-7b5ec6eaa3c2"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:10:13.110509 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.110482 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73805294-484d-40d9-87ce-7b5ec6eaa3c2-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "73805294-484d-40d9-87ce-7b5ec6eaa3c2" (UID: "73805294-484d-40d9-87ce-7b5ec6eaa3c2"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:10:13.112678 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.112631 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "73805294-484d-40d9-87ce-7b5ec6eaa3c2" (UID: "73805294-484d-40d9-87ce-7b5ec6eaa3c2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 13:10:13.113082 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.112973 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "73805294-484d-40d9-87ce-7b5ec6eaa3c2" (UID: "73805294-484d-40d9-87ce-7b5ec6eaa3c2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 13:10:13.114714 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.114686 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-config-volume" (OuterVolumeSpecName: "config-volume") pod "73805294-484d-40d9-87ce-7b5ec6eaa3c2" (UID: "73805294-484d-40d9-87ce-7b5ec6eaa3c2"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 13:10:13.114861 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.114816 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "73805294-484d-40d9-87ce-7b5ec6eaa3c2" (UID: "73805294-484d-40d9-87ce-7b5ec6eaa3c2"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 13:10:13.115027 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.114980 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "73805294-484d-40d9-87ce-7b5ec6eaa3c2" (UID: "73805294-484d-40d9-87ce-7b5ec6eaa3c2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 13:10:13.115027 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.115014 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73805294-484d-40d9-87ce-7b5ec6eaa3c2-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "73805294-484d-40d9-87ce-7b5ec6eaa3c2" (UID: "73805294-484d-40d9-87ce-7b5ec6eaa3c2"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:10:13.115170 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.115045 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73805294-484d-40d9-87ce-7b5ec6eaa3c2-config-out" (OuterVolumeSpecName: "config-out") pod "73805294-484d-40d9-87ce-7b5ec6eaa3c2" (UID: "73805294-484d-40d9-87ce-7b5ec6eaa3c2"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:10:13.115170 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.115150 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73805294-484d-40d9-87ce-7b5ec6eaa3c2-kube-api-access-2mg8r" (OuterVolumeSpecName: "kube-api-access-2mg8r") pod "73805294-484d-40d9-87ce-7b5ec6eaa3c2" (UID: "73805294-484d-40d9-87ce-7b5ec6eaa3c2"). InnerVolumeSpecName "kube-api-access-2mg8r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:10:13.118523 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.118221 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "73805294-484d-40d9-87ce-7b5ec6eaa3c2" (UID: "73805294-484d-40d9-87ce-7b5ec6eaa3c2"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 13:10:13.124498 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.124475 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-web-config" (OuterVolumeSpecName: "web-config") pod "73805294-484d-40d9-87ce-7b5ec6eaa3c2" (UID: "73805294-484d-40d9-87ce-7b5ec6eaa3c2"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 13:10:13.196603 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.196518 2531 generic.go:356] "Generic (PLEG): container finished" podID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerID="568d77d5e067b753efb61217378d735e64238f70026787e1426407da9236d023" exitCode=0 Jul 23 13:10:13.196603 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.196551 2531 generic.go:356] "Generic (PLEG): container finished" podID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerID="39c51f6bdf24f3153d90875a23be3eca104679cb3df0fb9dc67c6218685a45e8" exitCode=0 Jul 23 13:10:13.196603 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.196584 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73805294-484d-40d9-87ce-7b5ec6eaa3c2","Type":"ContainerDied","Data":"568d77d5e067b753efb61217378d735e64238f70026787e1426407da9236d023"} Jul 23 13:10:13.196797 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.196607 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.196797 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.196640 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73805294-484d-40d9-87ce-7b5ec6eaa3c2","Type":"ContainerDied","Data":"39c51f6bdf24f3153d90875a23be3eca104679cb3df0fb9dc67c6218685a45e8"} Jul 23 13:10:13.196797 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.196668 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"73805294-484d-40d9-87ce-7b5ec6eaa3c2","Type":"ContainerDied","Data":"55da06a65b7fff8ab71e03a3c20f54ac13e8f5e3213f8e8405e88e7a24f72c17"} Jul 23 13:10:13.196797 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.196730 2531 scope.go:117] "RemoveContainer" containerID="0c507f18ba55ff4a23504d9894a4be93cefa734e6d16af6792a67c7875d8e550" Jul 23 13:10:13.204319 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.204294 2531 scope.go:117] "RemoveContainer" containerID="568d77d5e067b753efb61217378d735e64238f70026787e1426407da9236d023" Jul 23 13:10:13.210993 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.210810 2531 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/73805294-484d-40d9-87ce-7b5ec6eaa3c2-config-out\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:10:13.210993 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.210837 2531 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:10:13.210993 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.210850 2531 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:10:13.210993 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.210864 2531 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-main-tls\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:10:13.210993 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.210876 2531 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-cluster-tls-config\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:10:13.210993 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.210888 2531 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73805294-484d-40d9-87ce-7b5ec6eaa3c2-metrics-client-ca\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:10:13.210993 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.210901 2531 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-web-config\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:10:13.210993 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.210912 2531 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:10:13.210993 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.210926 2531 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/73805294-484d-40d9-87ce-7b5ec6eaa3c2-tls-assets\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:10:13.210993 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.210938 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mg8r\" (UniqueName: \"kubernetes.io/projected/73805294-484d-40d9-87ce-7b5ec6eaa3c2-kube-api-access-2mg8r\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:10:13.210993 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.210949 2531 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/73805294-484d-40d9-87ce-7b5ec6eaa3c2-config-volume\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:10:13.210993 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.210961 2531 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73805294-484d-40d9-87ce-7b5ec6eaa3c2-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:10:13.210993 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.210973 2531 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/73805294-484d-40d9-87ce-7b5ec6eaa3c2-alertmanager-main-db\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:10:13.212968 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.212948 2531 scope.go:117] "RemoveContainer" containerID="e04cb036475a8027f4724984710af645bcafd63f96155078f053876f9fb42262" Jul 23 13:10:13.219918 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.219895 2531 scope.go:117] "RemoveContainer" containerID="39c51f6bdf24f3153d90875a23be3eca104679cb3df0fb9dc67c6218685a45e8" Jul 23 13:10:13.220981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.220965 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 13:10:13.225057 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.225038 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 13:10:13.227273 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.227256 2531 scope.go:117] "RemoveContainer" containerID="4d71534fc988a00bb0a8d73ad5479e7bca610255ed7347cfd0958d4074fe1f51" Jul 23 13:10:13.234028 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.234013 2531 scope.go:117] "RemoveContainer" containerID="aa545e9d5b0b24c6676a6ed9b1df1c12e198f9b3ff1022fbe17927e0bb4d37c9" Jul 23 13:10:13.241208 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.241185 2531 scope.go:117] "RemoveContainer" containerID="709bc0ed85e5eeae1a25108354922726bb0bf898cd0fc41baef514e0fc22d401" Jul 23 13:10:13.248839 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.248818 2531 scope.go:117] "RemoveContainer" containerID="0c507f18ba55ff4a23504d9894a4be93cefa734e6d16af6792a67c7875d8e550" Jul 23 13:10:13.249077 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:10:13.249055 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c507f18ba55ff4a23504d9894a4be93cefa734e6d16af6792a67c7875d8e550\": container with ID starting with 0c507f18ba55ff4a23504d9894a4be93cefa734e6d16af6792a67c7875d8e550 not found: ID does not exist" containerID="0c507f18ba55ff4a23504d9894a4be93cefa734e6d16af6792a67c7875d8e550" Jul 23 13:10:13.249152 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.249087 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c507f18ba55ff4a23504d9894a4be93cefa734e6d16af6792a67c7875d8e550"} err="failed to get container status \"0c507f18ba55ff4a23504d9894a4be93cefa734e6d16af6792a67c7875d8e550\": rpc error: code = NotFound desc = could not find container \"0c507f18ba55ff4a23504d9894a4be93cefa734e6d16af6792a67c7875d8e550\": container with ID starting with 0c507f18ba55ff4a23504d9894a4be93cefa734e6d16af6792a67c7875d8e550 not found: ID does not exist" Jul 23 13:10:13.249152 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.249130 2531 scope.go:117] "RemoveContainer" containerID="568d77d5e067b753efb61217378d735e64238f70026787e1426407da9236d023" Jul 23 13:10:13.249366 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:10:13.249330 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"568d77d5e067b753efb61217378d735e64238f70026787e1426407da9236d023\": container with ID starting with 568d77d5e067b753efb61217378d735e64238f70026787e1426407da9236d023 not found: ID does not exist" containerID="568d77d5e067b753efb61217378d735e64238f70026787e1426407da9236d023" Jul 23 13:10:13.249424 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.249380 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"568d77d5e067b753efb61217378d735e64238f70026787e1426407da9236d023"} err="failed to get container status \"568d77d5e067b753efb61217378d735e64238f70026787e1426407da9236d023\": rpc error: code = NotFound desc = could not find container \"568d77d5e067b753efb61217378d735e64238f70026787e1426407da9236d023\": container with ID starting with 568d77d5e067b753efb61217378d735e64238f70026787e1426407da9236d023 not found: ID does not exist" Jul 23 13:10:13.249424 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.249398 2531 scope.go:117] "RemoveContainer" containerID="e04cb036475a8027f4724984710af645bcafd63f96155078f053876f9fb42262" Jul 23 13:10:13.249576 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:10:13.249557 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e04cb036475a8027f4724984710af645bcafd63f96155078f053876f9fb42262\": container with ID starting with e04cb036475a8027f4724984710af645bcafd63f96155078f053876f9fb42262 not found: ID does not exist" containerID="e04cb036475a8027f4724984710af645bcafd63f96155078f053876f9fb42262" Jul 23 13:10:13.249626 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.249581 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e04cb036475a8027f4724984710af645bcafd63f96155078f053876f9fb42262"} err="failed to get container status \"e04cb036475a8027f4724984710af645bcafd63f96155078f053876f9fb42262\": rpc error: code = NotFound desc = could not find container \"e04cb036475a8027f4724984710af645bcafd63f96155078f053876f9fb42262\": container with ID starting with e04cb036475a8027f4724984710af645bcafd63f96155078f053876f9fb42262 not found: ID does not exist" Jul 23 13:10:13.249626 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.249600 2531 scope.go:117] "RemoveContainer" containerID="39c51f6bdf24f3153d90875a23be3eca104679cb3df0fb9dc67c6218685a45e8" Jul 23 13:10:13.249792 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:10:13.249777 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39c51f6bdf24f3153d90875a23be3eca104679cb3df0fb9dc67c6218685a45e8\": container with ID starting with 39c51f6bdf24f3153d90875a23be3eca104679cb3df0fb9dc67c6218685a45e8 not found: ID does not exist" containerID="39c51f6bdf24f3153d90875a23be3eca104679cb3df0fb9dc67c6218685a45e8" Jul 23 13:10:13.249832 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.249795 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39c51f6bdf24f3153d90875a23be3eca104679cb3df0fb9dc67c6218685a45e8"} err="failed to get container status \"39c51f6bdf24f3153d90875a23be3eca104679cb3df0fb9dc67c6218685a45e8\": rpc error: code = NotFound desc = could not find container \"39c51f6bdf24f3153d90875a23be3eca104679cb3df0fb9dc67c6218685a45e8\": container with ID starting with 39c51f6bdf24f3153d90875a23be3eca104679cb3df0fb9dc67c6218685a45e8 not found: ID does not exist" Jul 23 13:10:13.249832 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.249808 2531 scope.go:117] "RemoveContainer" containerID="4d71534fc988a00bb0a8d73ad5479e7bca610255ed7347cfd0958d4074fe1f51" Jul 23 13:10:13.249981 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:10:13.249959 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d71534fc988a00bb0a8d73ad5479e7bca610255ed7347cfd0958d4074fe1f51\": container with ID starting with 4d71534fc988a00bb0a8d73ad5479e7bca610255ed7347cfd0958d4074fe1f51 not found: ID does not exist" containerID="4d71534fc988a00bb0a8d73ad5479e7bca610255ed7347cfd0958d4074fe1f51" Jul 23 13:10:13.250020 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.249985 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d71534fc988a00bb0a8d73ad5479e7bca610255ed7347cfd0958d4074fe1f51"} err="failed to get container status \"4d71534fc988a00bb0a8d73ad5479e7bca610255ed7347cfd0958d4074fe1f51\": rpc error: code = NotFound desc = could not find container \"4d71534fc988a00bb0a8d73ad5479e7bca610255ed7347cfd0958d4074fe1f51\": container with ID starting with 4d71534fc988a00bb0a8d73ad5479e7bca610255ed7347cfd0958d4074fe1f51 not found: ID does not exist" Jul 23 13:10:13.250020 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.249998 2531 scope.go:117] "RemoveContainer" containerID="aa545e9d5b0b24c6676a6ed9b1df1c12e198f9b3ff1022fbe17927e0bb4d37c9" Jul 23 13:10:13.250191 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:10:13.250177 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa545e9d5b0b24c6676a6ed9b1df1c12e198f9b3ff1022fbe17927e0bb4d37c9\": container with ID starting with aa545e9d5b0b24c6676a6ed9b1df1c12e198f9b3ff1022fbe17927e0bb4d37c9 not found: ID does not exist" containerID="aa545e9d5b0b24c6676a6ed9b1df1c12e198f9b3ff1022fbe17927e0bb4d37c9" Jul 23 13:10:13.250232 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.250193 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa545e9d5b0b24c6676a6ed9b1df1c12e198f9b3ff1022fbe17927e0bb4d37c9"} err="failed to get container status \"aa545e9d5b0b24c6676a6ed9b1df1c12e198f9b3ff1022fbe17927e0bb4d37c9\": rpc error: code = NotFound desc = could not find container \"aa545e9d5b0b24c6676a6ed9b1df1c12e198f9b3ff1022fbe17927e0bb4d37c9\": container with ID starting with aa545e9d5b0b24c6676a6ed9b1df1c12e198f9b3ff1022fbe17927e0bb4d37c9 not found: ID does not exist" Jul 23 13:10:13.250232 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.250204 2531 scope.go:117] "RemoveContainer" containerID="709bc0ed85e5eeae1a25108354922726bb0bf898cd0fc41baef514e0fc22d401" Jul 23 13:10:13.250406 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:10:13.250388 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"709bc0ed85e5eeae1a25108354922726bb0bf898cd0fc41baef514e0fc22d401\": container with ID starting with 709bc0ed85e5eeae1a25108354922726bb0bf898cd0fc41baef514e0fc22d401 not found: ID does not exist" containerID="709bc0ed85e5eeae1a25108354922726bb0bf898cd0fc41baef514e0fc22d401" Jul 23 13:10:13.250471 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.250412 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"709bc0ed85e5eeae1a25108354922726bb0bf898cd0fc41baef514e0fc22d401"} err="failed to get container status \"709bc0ed85e5eeae1a25108354922726bb0bf898cd0fc41baef514e0fc22d401\": rpc error: code = NotFound desc = could not find container \"709bc0ed85e5eeae1a25108354922726bb0bf898cd0fc41baef514e0fc22d401\": container with ID starting with 709bc0ed85e5eeae1a25108354922726bb0bf898cd0fc41baef514e0fc22d401 not found: ID does not exist" Jul 23 13:10:13.250471 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.250433 2531 scope.go:117] "RemoveContainer" containerID="0c507f18ba55ff4a23504d9894a4be93cefa734e6d16af6792a67c7875d8e550" Jul 23 13:10:13.250632 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.250615 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c507f18ba55ff4a23504d9894a4be93cefa734e6d16af6792a67c7875d8e550"} err="failed to get container status \"0c507f18ba55ff4a23504d9894a4be93cefa734e6d16af6792a67c7875d8e550\": rpc error: code = NotFound desc = could not find container \"0c507f18ba55ff4a23504d9894a4be93cefa734e6d16af6792a67c7875d8e550\": container with ID starting with 0c507f18ba55ff4a23504d9894a4be93cefa734e6d16af6792a67c7875d8e550 not found: ID does not exist" Jul 23 13:10:13.250675 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.250632 2531 scope.go:117] "RemoveContainer" containerID="568d77d5e067b753efb61217378d735e64238f70026787e1426407da9236d023" Jul 23 13:10:13.250916 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.250890 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"568d77d5e067b753efb61217378d735e64238f70026787e1426407da9236d023"} err="failed to get container status \"568d77d5e067b753efb61217378d735e64238f70026787e1426407da9236d023\": rpc error: code = NotFound desc = could not find container \"568d77d5e067b753efb61217378d735e64238f70026787e1426407da9236d023\": container with ID starting with 568d77d5e067b753efb61217378d735e64238f70026787e1426407da9236d023 not found: ID does not exist" Jul 23 13:10:13.251017 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.250917 2531 scope.go:117] "RemoveContainer" containerID="e04cb036475a8027f4724984710af645bcafd63f96155078f053876f9fb42262" Jul 23 13:10:13.251298 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.251274 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e04cb036475a8027f4724984710af645bcafd63f96155078f053876f9fb42262"} err="failed to get container status \"e04cb036475a8027f4724984710af645bcafd63f96155078f053876f9fb42262\": rpc error: code = NotFound desc = could not find container \"e04cb036475a8027f4724984710af645bcafd63f96155078f053876f9fb42262\": container with ID starting with e04cb036475a8027f4724984710af645bcafd63f96155078f053876f9fb42262 not found: ID does not exist" Jul 23 13:10:13.251405 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.251299 2531 scope.go:117] "RemoveContainer" containerID="39c51f6bdf24f3153d90875a23be3eca104679cb3df0fb9dc67c6218685a45e8" Jul 23 13:10:13.251630 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.251601 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39c51f6bdf24f3153d90875a23be3eca104679cb3df0fb9dc67c6218685a45e8"} err="failed to get container status \"39c51f6bdf24f3153d90875a23be3eca104679cb3df0fb9dc67c6218685a45e8\": rpc error: code = NotFound desc = could not find container \"39c51f6bdf24f3153d90875a23be3eca104679cb3df0fb9dc67c6218685a45e8\": container with ID starting with 39c51f6bdf24f3153d90875a23be3eca104679cb3df0fb9dc67c6218685a45e8 not found: ID does not exist" Jul 23 13:10:13.251745 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.251631 2531 scope.go:117] "RemoveContainer" containerID="4d71534fc988a00bb0a8d73ad5479e7bca610255ed7347cfd0958d4074fe1f51" Jul 23 13:10:13.251990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.251969 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d71534fc988a00bb0a8d73ad5479e7bca610255ed7347cfd0958d4074fe1f51"} err="failed to get container status \"4d71534fc988a00bb0a8d73ad5479e7bca610255ed7347cfd0958d4074fe1f51\": rpc error: code = NotFound desc = could not find container \"4d71534fc988a00bb0a8d73ad5479e7bca610255ed7347cfd0958d4074fe1f51\": container with ID starting with 4d71534fc988a00bb0a8d73ad5479e7bca610255ed7347cfd0958d4074fe1f51 not found: ID does not exist" Jul 23 13:10:13.252069 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.251993 2531 scope.go:117] "RemoveContainer" containerID="aa545e9d5b0b24c6676a6ed9b1df1c12e198f9b3ff1022fbe17927e0bb4d37c9" Jul 23 13:10:13.252263 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.252244 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa545e9d5b0b24c6676a6ed9b1df1c12e198f9b3ff1022fbe17927e0bb4d37c9"} err="failed to get container status \"aa545e9d5b0b24c6676a6ed9b1df1c12e198f9b3ff1022fbe17927e0bb4d37c9\": rpc error: code = NotFound desc = could not find container \"aa545e9d5b0b24c6676a6ed9b1df1c12e198f9b3ff1022fbe17927e0bb4d37c9\": container with ID starting with aa545e9d5b0b24c6676a6ed9b1df1c12e198f9b3ff1022fbe17927e0bb4d37c9 not found: ID does not exist" Jul 23 13:10:13.252263 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.252262 2531 scope.go:117] "RemoveContainer" containerID="709bc0ed85e5eeae1a25108354922726bb0bf898cd0fc41baef514e0fc22d401" Jul 23 13:10:13.252564 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.252533 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"709bc0ed85e5eeae1a25108354922726bb0bf898cd0fc41baef514e0fc22d401"} err="failed to get container status \"709bc0ed85e5eeae1a25108354922726bb0bf898cd0fc41baef514e0fc22d401\": rpc error: code = NotFound desc = could not find container \"709bc0ed85e5eeae1a25108354922726bb0bf898cd0fc41baef514e0fc22d401\": container with ID starting with 709bc0ed85e5eeae1a25108354922726bb0bf898cd0fc41baef514e0fc22d401 not found: ID does not exist" Jul 23 13:10:13.252806 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.252793 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 13:10:13.253052 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.253041 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="kube-rbac-proxy" Jul 23 13:10:13.253086 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.253055 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="kube-rbac-proxy" Jul 23 13:10:13.253086 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.253065 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="config-reloader" Jul 23 13:10:13.253086 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.253070 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="config-reloader" Jul 23 13:10:13.253086 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.253077 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="init-config-reloader" Jul 23 13:10:13.253086 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.253083 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="init-config-reloader" Jul 23 13:10:13.253223 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.253093 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="kube-rbac-proxy-web" Jul 23 13:10:13.253223 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.253099 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="kube-rbac-proxy-web" Jul 23 13:10:13.253223 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.253106 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="alertmanager" Jul 23 13:10:13.253223 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.253111 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="alertmanager" Jul 23 13:10:13.253223 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.253118 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="prom-label-proxy" Jul 23 13:10:13.253223 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.253123 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="prom-label-proxy" Jul 23 13:10:13.253223 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.253132 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="kube-rbac-proxy-metric" Jul 23 13:10:13.253223 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.253137 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="kube-rbac-proxy-metric" Jul 23 13:10:13.253223 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.253173 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="config-reloader" Jul 23 13:10:13.253223 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.253181 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="kube-rbac-proxy-web" Jul 23 13:10:13.253223 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.253187 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="kube-rbac-proxy-metric" Jul 23 13:10:13.253223 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.253192 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="alertmanager" Jul 23 13:10:13.253223 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.253197 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="kube-rbac-proxy" Jul 23 13:10:13.253223 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.253201 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" containerName="prom-label-proxy" Jul 23 13:10:13.256837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.256822 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.259149 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.259116 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 23 13:10:13.259226 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.259116 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 23 13:10:13.259226 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.259179 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 23 13:10:13.259294 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.259212 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 23 13:10:13.259294 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.259124 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 23 13:10:13.259294 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.259225 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 23 13:10:13.259529 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.259514 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-v7mzp\"" Jul 23 13:10:13.259570 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.259560 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 23 13:10:13.259609 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.259573 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 23 13:10:13.265936 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.265901 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 23 13:10:13.270496 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.270477 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 13:10:13.311395 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.311345 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.311546 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.311406 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-web-config\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.311546 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.311458 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-config-volume\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.311546 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.311492 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.311546 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.311527 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.311702 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.311548 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.311702 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.311606 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.311702 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.311643 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.311702 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.311662 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxwsf\" (UniqueName: \"kubernetes.io/projected/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-kube-api-access-sxwsf\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.311702 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.311690 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-config-out\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.311902 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.311713 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.311902 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.311749 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.311902 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.311795 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.334985 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.334955 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73805294-484d-40d9-87ce-7b5ec6eaa3c2" path="/var/lib/kubelet/pods/73805294-484d-40d9-87ce-7b5ec6eaa3c2/volumes" Jul 23 13:10:13.412770 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.412732 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.412770 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.412773 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.412965 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.412791 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxwsf\" (UniqueName: \"kubernetes.io/projected/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-kube-api-access-sxwsf\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.412965 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.412817 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-config-out\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.412965 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.412836 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.412965 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.412851 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.412965 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.412869 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.412965 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.412889 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.412965 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.412906 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-web-config\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.412965 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.412933 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-config-volume\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.413341 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.412970 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.413341 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.413011 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.413341 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.413036 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.413341 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.413299 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.414111 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.413639 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.414309 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.414282 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.416012 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.415983 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-config-out\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.416235 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.416207 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.416317 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.416291 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-config-volume\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.416395 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.416326 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.416395 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.416346 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.416512 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.416394 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.416579 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.416553 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-web-config\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.417190 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.417170 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.418013 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.417996 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.421136 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.421120 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxwsf\" (UniqueName: \"kubernetes.io/projected/5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1-kube-api-access-sxwsf\") pod \"alertmanager-main-0\" (UID: \"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.566214 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.566182 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 23 13:10:13.703224 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:13.703201 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 13:10:13.704894 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:10:13.704867 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5670b455_1fa3_4e4c_8ff1_0ff3ec48a7c1.slice/crio-e73692c1debef3e6b6b973dab8c34bb5151066e53335280296a7ba578a4bb661 WatchSource:0}: Error finding container e73692c1debef3e6b6b973dab8c34bb5151066e53335280296a7ba578a4bb661: Status 404 returned error can't find the container with id e73692c1debef3e6b6b973dab8c34bb5151066e53335280296a7ba578a4bb661 Jul 23 13:10:14.203088 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:14.203000 2531 generic.go:356] "Generic (PLEG): container finished" podID="5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1" containerID="7d7b55f0bfe2740c2fb67878255a279543ddd09fd700a0eda685da22ae94022c" exitCode=0 Jul 23 13:10:14.203544 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:14.203083 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1","Type":"ContainerDied","Data":"7d7b55f0bfe2740c2fb67878255a279543ddd09fd700a0eda685da22ae94022c"} Jul 23 13:10:14.203544 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:14.203167 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1","Type":"ContainerStarted","Data":"e73692c1debef3e6b6b973dab8c34bb5151066e53335280296a7ba578a4bb661"} Jul 23 13:10:15.210758 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:15.210723 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1","Type":"ContainerStarted","Data":"dc1fb14dabc33338c0c1c8dfeb72d583bd1167bfe9aad347956bbc92aa91cbe4"} Jul 23 13:10:15.211129 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:15.210787 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1","Type":"ContainerStarted","Data":"99b8a7785ab787bc1d6c2a6966848db211bcc505f6296bf999b463ad8c6b9a5f"} Jul 23 13:10:15.211129 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:15.210811 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1","Type":"ContainerStarted","Data":"88f0bc2cc485da790103ab16e71ef346aabd1b5726afd392a5d3966a9af73776"} Jul 23 13:10:15.211129 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:15.210837 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1","Type":"ContainerStarted","Data":"0261f813b55d145df5c7a907faa201e5ddf0e5ab46e6a76c12c8d47c548fdfe0"} Jul 23 13:10:15.211129 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:15.210863 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1","Type":"ContainerStarted","Data":"c82d67d819b9738da388b608d136895fc2f502c094c04fb1ece4c6f86e6744e2"} Jul 23 13:10:15.211129 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:15.210886 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1","Type":"ContainerStarted","Data":"5e79241cf0272e547ed8dd339c9ca294ec262300076056f63c3b37c8dbcc0ef6"} Jul 23 13:10:15.238691 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:15.238647 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.219734282 podStartE2EDuration="2.238630257s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.895975ms" isStatefulPod=false podCreationTimestamp="2026-07-23 13:10:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:10:15.236517655 +0000 UTC m=+152.475793633" watchObservedRunningTime="2026-07-23 13:10:15.238630257 +0000 UTC m=+152.477906234" Jul 23 13:10:15.822990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:15.822955 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-b786d8cbb-vc7wp"] Jul 23 13:10:15.825327 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:15.825310 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:15.827690 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:15.827659 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-ldrmm\"" Jul 23 13:10:15.827818 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:15.827704 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 23 13:10:15.828017 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:15.828000 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 23 13:10:15.828096 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:15.828055 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 23 13:10:15.828150 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:15.828128 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 23 13:10:15.828289 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:15.828261 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 23 13:10:15.833659 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:15.833623 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 23 13:10:15.840156 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:15.840134 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-b786d8cbb-vc7wp"] Jul 23 13:10:15.931394 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:15.931334 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ace5549c-7111-4c03-8b91-a0f5c4217529-telemeter-trusted-ca-bundle\") pod \"telemeter-client-b786d8cbb-vc7wp\" (UID: \"ace5549c-7111-4c03-8b91-a0f5c4217529\") " pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:15.931564 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:15.931409 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ace5549c-7111-4c03-8b91-a0f5c4217529-telemeter-client-tls\") pod \"telemeter-client-b786d8cbb-vc7wp\" (UID: \"ace5549c-7111-4c03-8b91-a0f5c4217529\") " pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:15.931564 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:15.931429 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ace5549c-7111-4c03-8b91-a0f5c4217529-metrics-client-ca\") pod \"telemeter-client-b786d8cbb-vc7wp\" (UID: \"ace5549c-7111-4c03-8b91-a0f5c4217529\") " pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:15.931564 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:15.931446 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j7g8\" (UniqueName: \"kubernetes.io/projected/ace5549c-7111-4c03-8b91-a0f5c4217529-kube-api-access-8j7g8\") pod \"telemeter-client-b786d8cbb-vc7wp\" (UID: \"ace5549c-7111-4c03-8b91-a0f5c4217529\") " pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:15.931564 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:15.931464 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ace5549c-7111-4c03-8b91-a0f5c4217529-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-b786d8cbb-vc7wp\" (UID: \"ace5549c-7111-4c03-8b91-a0f5c4217529\") " pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:15.931564 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:15.931484 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ace5549c-7111-4c03-8b91-a0f5c4217529-serving-certs-ca-bundle\") pod \"telemeter-client-b786d8cbb-vc7wp\" (UID: \"ace5549c-7111-4c03-8b91-a0f5c4217529\") " pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:15.931564 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:15.931531 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ace5549c-7111-4c03-8b91-a0f5c4217529-secret-telemeter-client\") pod \"telemeter-client-b786d8cbb-vc7wp\" (UID: \"ace5549c-7111-4c03-8b91-a0f5c4217529\") " pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:15.931770 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:15.931574 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ace5549c-7111-4c03-8b91-a0f5c4217529-federate-client-tls\") pod \"telemeter-client-b786d8cbb-vc7wp\" (UID: \"ace5549c-7111-4c03-8b91-a0f5c4217529\") " pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:16.032721 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:16.032688 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ace5549c-7111-4c03-8b91-a0f5c4217529-serving-certs-ca-bundle\") pod \"telemeter-client-b786d8cbb-vc7wp\" (UID: \"ace5549c-7111-4c03-8b91-a0f5c4217529\") " pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:16.032721 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:16.032723 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ace5549c-7111-4c03-8b91-a0f5c4217529-secret-telemeter-client\") pod \"telemeter-client-b786d8cbb-vc7wp\" (UID: \"ace5549c-7111-4c03-8b91-a0f5c4217529\") " pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:16.032921 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:16.032753 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ace5549c-7111-4c03-8b91-a0f5c4217529-federate-client-tls\") pod \"telemeter-client-b786d8cbb-vc7wp\" (UID: \"ace5549c-7111-4c03-8b91-a0f5c4217529\") " pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:16.032921 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:16.032782 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ace5549c-7111-4c03-8b91-a0f5c4217529-telemeter-trusted-ca-bundle\") pod \"telemeter-client-b786d8cbb-vc7wp\" (UID: \"ace5549c-7111-4c03-8b91-a0f5c4217529\") " pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:16.032921 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:16.032819 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ace5549c-7111-4c03-8b91-a0f5c4217529-telemeter-client-tls\") pod \"telemeter-client-b786d8cbb-vc7wp\" (UID: \"ace5549c-7111-4c03-8b91-a0f5c4217529\") " pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:16.032921 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:16.032835 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ace5549c-7111-4c03-8b91-a0f5c4217529-metrics-client-ca\") pod \"telemeter-client-b786d8cbb-vc7wp\" (UID: \"ace5549c-7111-4c03-8b91-a0f5c4217529\") " pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:16.032921 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:16.032853 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j7g8\" (UniqueName: \"kubernetes.io/projected/ace5549c-7111-4c03-8b91-a0f5c4217529-kube-api-access-8j7g8\") pod \"telemeter-client-b786d8cbb-vc7wp\" (UID: \"ace5549c-7111-4c03-8b91-a0f5c4217529\") " pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:16.032921 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:16.032873 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ace5549c-7111-4c03-8b91-a0f5c4217529-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-b786d8cbb-vc7wp\" (UID: \"ace5549c-7111-4c03-8b91-a0f5c4217529\") " pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:16.033814 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:16.033788 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ace5549c-7111-4c03-8b91-a0f5c4217529-telemeter-trusted-ca-bundle\") pod \"telemeter-client-b786d8cbb-vc7wp\" (UID: \"ace5549c-7111-4c03-8b91-a0f5c4217529\") " pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:16.033929 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:16.033809 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ace5549c-7111-4c03-8b91-a0f5c4217529-metrics-client-ca\") pod \"telemeter-client-b786d8cbb-vc7wp\" (UID: \"ace5549c-7111-4c03-8b91-a0f5c4217529\") " pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:16.033929 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:16.033788 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ace5549c-7111-4c03-8b91-a0f5c4217529-serving-certs-ca-bundle\") pod \"telemeter-client-b786d8cbb-vc7wp\" (UID: \"ace5549c-7111-4c03-8b91-a0f5c4217529\") " pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:16.035433 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:16.035409 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ace5549c-7111-4c03-8b91-a0f5c4217529-telemeter-client-tls\") pod \"telemeter-client-b786d8cbb-vc7wp\" (UID: \"ace5549c-7111-4c03-8b91-a0f5c4217529\") " pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:16.035547 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:16.035443 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ace5549c-7111-4c03-8b91-a0f5c4217529-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-b786d8cbb-vc7wp\" (UID: \"ace5549c-7111-4c03-8b91-a0f5c4217529\") " pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:16.036174 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:16.036155 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ace5549c-7111-4c03-8b91-a0f5c4217529-secret-telemeter-client\") pod \"telemeter-client-b786d8cbb-vc7wp\" (UID: \"ace5549c-7111-4c03-8b91-a0f5c4217529\") " pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:16.036327 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:16.036308 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ace5549c-7111-4c03-8b91-a0f5c4217529-federate-client-tls\") pod \"telemeter-client-b786d8cbb-vc7wp\" (UID: \"ace5549c-7111-4c03-8b91-a0f5c4217529\") " pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:16.040880 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:16.040857 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j7g8\" (UniqueName: \"kubernetes.io/projected/ace5549c-7111-4c03-8b91-a0f5c4217529-kube-api-access-8j7g8\") pod \"telemeter-client-b786d8cbb-vc7wp\" (UID: \"ace5549c-7111-4c03-8b91-a0f5c4217529\") " pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:16.136552 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:16.136459 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" Jul 23 13:10:16.268802 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:16.268774 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-b786d8cbb-vc7wp"] Jul 23 13:10:16.271803 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:10:16.271773 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podace5549c_7111_4c03_8b91_a0f5c4217529.slice/crio-82a77dd34aa4017c4881324a8e5b723ba5d144f588de99e54ec7ba4c3bf2c671 WatchSource:0}: Error finding container 82a77dd34aa4017c4881324a8e5b723ba5d144f588de99e54ec7ba4c3bf2c671: Status 404 returned error can't find the container with id 82a77dd34aa4017c4881324a8e5b723ba5d144f588de99e54ec7ba4c3bf2c671 Jul 23 13:10:17.221885 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:17.221854 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" event={"ID":"ace5549c-7111-4c03-8b91-a0f5c4217529","Type":"ContainerStarted","Data":"82a77dd34aa4017c4881324a8e5b723ba5d144f588de99e54ec7ba4c3bf2c671"} Jul 23 13:10:19.231741 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:19.231702 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" event={"ID":"ace5549c-7111-4c03-8b91-a0f5c4217529","Type":"ContainerStarted","Data":"65b1dabf3c3ddd8e6978830f0ec4fc629adef79605150ab1c33e10da45333176"} Jul 23 13:10:19.232111 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:19.231755 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" event={"ID":"ace5549c-7111-4c03-8b91-a0f5c4217529","Type":"ContainerStarted","Data":"4f215e54edda31128c6fc9b44b1b11a3bd801582534e53bc8b9e40cb7f0a108e"} Jul 23 13:10:19.232111 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:19.231777 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" event={"ID":"ace5549c-7111-4c03-8b91-a0f5c4217529","Type":"ContainerStarted","Data":"40d066bd06cb8896d44c1fce86c921f1d9dbfc15b8dd3d7c3303e51d3f2b5f79"} Jul 23 13:10:19.253691 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:19.253640 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-b786d8cbb-vc7wp" podStartSLOduration=2.286430087 podStartE2EDuration="4.253624114s" totalImagesPullingTime="1.967194027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:10:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:10:19.251753609 +0000 UTC m=+156.491029584" watchObservedRunningTime="2026-07-23 13:10:19.253624114 +0000 UTC m=+156.492900089" Jul 23 13:10:19.979982 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:19.979947 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75bf6c86b4-dbrmr"] Jul 23 13:10:19.982271 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:19.982247 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:19.993576 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:19.993554 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75bf6c86b4-dbrmr"] Jul 23 13:10:20.066581 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:20.066542 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e1d9807-c931-4bc5-9127-3cf68e31b692-console-serving-cert\") pod \"console-75bf6c86b4-dbrmr\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:20.066581 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:20.066583 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e1d9807-c931-4bc5-9127-3cf68e31b692-oauth-serving-cert\") pod \"console-75bf6c86b4-dbrmr\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:20.066837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:20.066611 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e1d9807-c931-4bc5-9127-3cf68e31b692-service-ca\") pod \"console-75bf6c86b4-dbrmr\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:20.066837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:20.066630 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e1d9807-c931-4bc5-9127-3cf68e31b692-trusted-ca-bundle\") pod \"console-75bf6c86b4-dbrmr\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:20.066837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:20.066710 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e1d9807-c931-4bc5-9127-3cf68e31b692-console-config\") pod \"console-75bf6c86b4-dbrmr\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:20.066837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:20.066746 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e1d9807-c931-4bc5-9127-3cf68e31b692-console-oauth-config\") pod \"console-75bf6c86b4-dbrmr\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:20.066837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:20.066824 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vjfs\" (UniqueName: \"kubernetes.io/projected/6e1d9807-c931-4bc5-9127-3cf68e31b692-kube-api-access-8vjfs\") pod \"console-75bf6c86b4-dbrmr\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:20.168142 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:20.168105 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e1d9807-c931-4bc5-9127-3cf68e31b692-console-config\") pod \"console-75bf6c86b4-dbrmr\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:20.168142 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:20.168143 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e1d9807-c931-4bc5-9127-3cf68e31b692-console-oauth-config\") pod \"console-75bf6c86b4-dbrmr\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:20.168436 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:20.168176 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vjfs\" (UniqueName: \"kubernetes.io/projected/6e1d9807-c931-4bc5-9127-3cf68e31b692-kube-api-access-8vjfs\") pod \"console-75bf6c86b4-dbrmr\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:20.168436 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:20.168212 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e1d9807-c931-4bc5-9127-3cf68e31b692-console-serving-cert\") pod \"console-75bf6c86b4-dbrmr\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:20.168436 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:20.168228 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e1d9807-c931-4bc5-9127-3cf68e31b692-oauth-serving-cert\") pod \"console-75bf6c86b4-dbrmr\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:20.168436 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:20.168253 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e1d9807-c931-4bc5-9127-3cf68e31b692-service-ca\") pod \"console-75bf6c86b4-dbrmr\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:20.168436 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:20.168271 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e1d9807-c931-4bc5-9127-3cf68e31b692-trusted-ca-bundle\") pod \"console-75bf6c86b4-dbrmr\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:20.168994 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:20.168969 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e1d9807-c931-4bc5-9127-3cf68e31b692-service-ca\") pod \"console-75bf6c86b4-dbrmr\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:20.169106 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:20.169040 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e1d9807-c931-4bc5-9127-3cf68e31b692-oauth-serving-cert\") pod \"console-75bf6c86b4-dbrmr\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:20.169244 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:20.169223 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e1d9807-c931-4bc5-9127-3cf68e31b692-console-config\") pod \"console-75bf6c86b4-dbrmr\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:20.169574 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:20.169550 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e1d9807-c931-4bc5-9127-3cf68e31b692-trusted-ca-bundle\") pod \"console-75bf6c86b4-dbrmr\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:20.170904 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:20.170875 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e1d9807-c931-4bc5-9127-3cf68e31b692-console-serving-cert\") pod \"console-75bf6c86b4-dbrmr\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:20.170904 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:20.170889 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e1d9807-c931-4bc5-9127-3cf68e31b692-console-oauth-config\") pod \"console-75bf6c86b4-dbrmr\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:20.175699 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:20.175675 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vjfs\" (UniqueName: \"kubernetes.io/projected/6e1d9807-c931-4bc5-9127-3cf68e31b692-kube-api-access-8vjfs\") pod \"console-75bf6c86b4-dbrmr\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:20.291509 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:20.291425 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:20.424856 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:20.424823 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75bf6c86b4-dbrmr"] Jul 23 13:10:20.428152 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:10:20.428123 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e1d9807_c931_4bc5_9127_3cf68e31b692.slice/crio-131f91634be8206c7a6fe11cb08662ba660f19682b333145e8cab0fda054a54e WatchSource:0}: Error finding container 131f91634be8206c7a6fe11cb08662ba660f19682b333145e8cab0fda054a54e: Status 404 returned error can't find the container with id 131f91634be8206c7a6fe11cb08662ba660f19682b333145e8cab0fda054a54e Jul 23 13:10:21.241534 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:21.241495 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75bf6c86b4-dbrmr" event={"ID":"6e1d9807-c931-4bc5-9127-3cf68e31b692","Type":"ContainerStarted","Data":"acdddb5cd8cdd2dd49ef6f5e2163b3f786bc9b30d72e235e7b52b04989358026"} Jul 23 13:10:21.241730 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:21.241551 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75bf6c86b4-dbrmr" event={"ID":"6e1d9807-c931-4bc5-9127-3cf68e31b692","Type":"ContainerStarted","Data":"131f91634be8206c7a6fe11cb08662ba660f19682b333145e8cab0fda054a54e"} Jul 23 13:10:21.259633 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:21.259579 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75bf6c86b4-dbrmr" podStartSLOduration=2.259564181 podStartE2EDuration="2.259564181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:10:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:10:21.258390546 +0000 UTC m=+158.497666514" watchObservedRunningTime="2026-07-23 13:10:21.259564181 +0000 UTC m=+158.498840157" Jul 23 13:10:30.292528 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:30.292485 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:30.292946 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:30.292551 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:30.297236 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:30.297211 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:31.284058 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:31.284027 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:10:31.333434 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:31.333408 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5676dcddfb-8zklg"] Jul 23 13:10:56.354787 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:56.354743 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5676dcddfb-8zklg" podUID="3828321a-3058-4e5f-a076-3689380b9a9a" containerName="console" containerID="cri-o://f84bf8648197257d9438e29ffa30d8b49f404c93b7eece7546244e19f9ddaf48" gracePeriod=15 Jul 23 13:10:56.603885 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:56.603863 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:10:56.667739 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:56.667659 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3828321a-3058-4e5f-a076-3689380b9a9a-service-ca\") pod \"3828321a-3058-4e5f-a076-3689380b9a9a\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " Jul 23 13:10:56.667739 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:56.667712 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6p9q8\" (UniqueName: \"kubernetes.io/projected/3828321a-3058-4e5f-a076-3689380b9a9a-kube-api-access-6p9q8\") pod \"3828321a-3058-4e5f-a076-3689380b9a9a\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " Jul 23 13:10:56.667957 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:56.667742 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3828321a-3058-4e5f-a076-3689380b9a9a-console-serving-cert\") pod \"3828321a-3058-4e5f-a076-3689380b9a9a\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " Jul 23 13:10:56.667957 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:56.667806 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3828321a-3058-4e5f-a076-3689380b9a9a-oauth-serving-cert\") pod \"3828321a-3058-4e5f-a076-3689380b9a9a\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " Jul 23 13:10:56.667957 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:56.667858 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3828321a-3058-4e5f-a076-3689380b9a9a-trusted-ca-bundle\") pod \"3828321a-3058-4e5f-a076-3689380b9a9a\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " Jul 23 13:10:56.667957 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:56.667893 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3828321a-3058-4e5f-a076-3689380b9a9a-console-config\") pod \"3828321a-3058-4e5f-a076-3689380b9a9a\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " Jul 23 13:10:56.667957 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:56.667944 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3828321a-3058-4e5f-a076-3689380b9a9a-console-oauth-config\") pod \"3828321a-3058-4e5f-a076-3689380b9a9a\" (UID: \"3828321a-3058-4e5f-a076-3689380b9a9a\") " Jul 23 13:10:56.668159 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:56.668108 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3828321a-3058-4e5f-a076-3689380b9a9a-service-ca" (OuterVolumeSpecName: "service-ca") pod "3828321a-3058-4e5f-a076-3689380b9a9a" (UID: "3828321a-3058-4e5f-a076-3689380b9a9a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:10:56.668215 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:56.668182 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3828321a-3058-4e5f-a076-3689380b9a9a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3828321a-3058-4e5f-a076-3689380b9a9a" (UID: "3828321a-3058-4e5f-a076-3689380b9a9a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:10:56.668215 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:56.668206 2531 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3828321a-3058-4e5f-a076-3689380b9a9a-service-ca\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:10:56.668370 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:56.668323 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3828321a-3058-4e5f-a076-3689380b9a9a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3828321a-3058-4e5f-a076-3689380b9a9a" (UID: "3828321a-3058-4e5f-a076-3689380b9a9a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:10:56.668458 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:56.668433 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3828321a-3058-4e5f-a076-3689380b9a9a-console-config" (OuterVolumeSpecName: "console-config") pod "3828321a-3058-4e5f-a076-3689380b9a9a" (UID: "3828321a-3058-4e5f-a076-3689380b9a9a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:10:56.670130 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:56.670095 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3828321a-3058-4e5f-a076-3689380b9a9a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3828321a-3058-4e5f-a076-3689380b9a9a" (UID: "3828321a-3058-4e5f-a076-3689380b9a9a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 13:10:56.670210 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:56.670164 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3828321a-3058-4e5f-a076-3689380b9a9a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3828321a-3058-4e5f-a076-3689380b9a9a" (UID: "3828321a-3058-4e5f-a076-3689380b9a9a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 13:10:56.670210 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:56.670172 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3828321a-3058-4e5f-a076-3689380b9a9a-kube-api-access-6p9q8" (OuterVolumeSpecName: "kube-api-access-6p9q8") pod "3828321a-3058-4e5f-a076-3689380b9a9a" (UID: "3828321a-3058-4e5f-a076-3689380b9a9a"). InnerVolumeSpecName "kube-api-access-6p9q8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:10:56.769172 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:56.769145 2531 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3828321a-3058-4e5f-a076-3689380b9a9a-console-oauth-config\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:10:56.769172 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:56.769170 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6p9q8\" (UniqueName: \"kubernetes.io/projected/3828321a-3058-4e5f-a076-3689380b9a9a-kube-api-access-6p9q8\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:10:56.769371 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:56.769179 2531 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3828321a-3058-4e5f-a076-3689380b9a9a-console-serving-cert\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:10:56.769371 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:56.769188 2531 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3828321a-3058-4e5f-a076-3689380b9a9a-oauth-serving-cert\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:10:56.769371 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:56.769198 2531 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3828321a-3058-4e5f-a076-3689380b9a9a-trusted-ca-bundle\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:10:56.769371 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:56.769206 2531 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3828321a-3058-4e5f-a076-3689380b9a9a-console-config\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:10:57.373838 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:57.373801 2531 generic.go:356] "Generic (PLEG): container finished" podID="3828321a-3058-4e5f-a076-3689380b9a9a" containerID="f84bf8648197257d9438e29ffa30d8b49f404c93b7eece7546244e19f9ddaf48" exitCode=0 Jul 23 13:10:57.374265 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:57.373854 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5676dcddfb-8zklg" Jul 23 13:10:57.374265 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:57.373852 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5676dcddfb-8zklg" event={"ID":"3828321a-3058-4e5f-a076-3689380b9a9a","Type":"ContainerDied","Data":"f84bf8648197257d9438e29ffa30d8b49f404c93b7eece7546244e19f9ddaf48"} Jul 23 13:10:57.374265 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:57.373987 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5676dcddfb-8zklg" event={"ID":"3828321a-3058-4e5f-a076-3689380b9a9a","Type":"ContainerDied","Data":"8ef27ee361cb2c0c239e5738178fd34cf3067e0e0bc70200639bf44bd3556248"} Jul 23 13:10:57.374265 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:57.374025 2531 scope.go:117] "RemoveContainer" containerID="f84bf8648197257d9438e29ffa30d8b49f404c93b7eece7546244e19f9ddaf48" Jul 23 13:10:57.384260 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:57.384204 2531 scope.go:117] "RemoveContainer" containerID="f84bf8648197257d9438e29ffa30d8b49f404c93b7eece7546244e19f9ddaf48" Jul 23 13:10:57.384632 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:10:57.384604 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f84bf8648197257d9438e29ffa30d8b49f404c93b7eece7546244e19f9ddaf48\": container with ID starting with f84bf8648197257d9438e29ffa30d8b49f404c93b7eece7546244e19f9ddaf48 not found: ID does not exist" containerID="f84bf8648197257d9438e29ffa30d8b49f404c93b7eece7546244e19f9ddaf48" Jul 23 13:10:57.384747 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:57.384641 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f84bf8648197257d9438e29ffa30d8b49f404c93b7eece7546244e19f9ddaf48"} err="failed to get container status \"f84bf8648197257d9438e29ffa30d8b49f404c93b7eece7546244e19f9ddaf48\": rpc error: code = NotFound desc = could not find container \"f84bf8648197257d9438e29ffa30d8b49f404c93b7eece7546244e19f9ddaf48\": container with ID starting with f84bf8648197257d9438e29ffa30d8b49f404c93b7eece7546244e19f9ddaf48 not found: ID does not exist" Jul 23 13:10:57.388942 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:57.388920 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5676dcddfb-8zklg"] Jul 23 13:10:57.392207 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:57.392189 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5676dcddfb-8zklg"] Jul 23 13:10:59.330858 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:10:59.330823 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3828321a-3058-4e5f-a076-3689380b9a9a" path="/var/lib/kubelet/pods/3828321a-3058-4e5f-a076-3689380b9a9a/volumes" Jul 23 13:11:27.583683 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.583650 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6fff7954c7-48vgr"] Jul 23 13:11:27.584143 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.583936 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3828321a-3058-4e5f-a076-3689380b9a9a" containerName="console" Jul 23 13:11:27.584143 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.583946 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="3828321a-3058-4e5f-a076-3689380b9a9a" containerName="console" Jul 23 13:11:27.584143 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.583994 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="3828321a-3058-4e5f-a076-3689380b9a9a" containerName="console" Jul 23 13:11:27.587152 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.587134 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:27.601305 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.601278 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fff7954c7-48vgr"] Jul 23 13:11:27.712688 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.712644 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69b9013b-1695-4f0e-bc47-bbd8101610c8-console-oauth-config\") pod \"console-6fff7954c7-48vgr\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:27.712688 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.712693 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69b9013b-1695-4f0e-bc47-bbd8101610c8-oauth-serving-cert\") pod \"console-6fff7954c7-48vgr\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:27.712896 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.712722 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69b9013b-1695-4f0e-bc47-bbd8101610c8-console-config\") pod \"console-6fff7954c7-48vgr\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:27.712896 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.712740 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69b9013b-1695-4f0e-bc47-bbd8101610c8-trusted-ca-bundle\") pod \"console-6fff7954c7-48vgr\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:27.712896 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.712763 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69b9013b-1695-4f0e-bc47-bbd8101610c8-service-ca\") pod \"console-6fff7954c7-48vgr\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:27.712896 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.712790 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69b9013b-1695-4f0e-bc47-bbd8101610c8-console-serving-cert\") pod \"console-6fff7954c7-48vgr\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:27.712896 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.712809 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4blf\" (UniqueName: \"kubernetes.io/projected/69b9013b-1695-4f0e-bc47-bbd8101610c8-kube-api-access-s4blf\") pod \"console-6fff7954c7-48vgr\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:27.813688 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.813651 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69b9013b-1695-4f0e-bc47-bbd8101610c8-console-oauth-config\") pod \"console-6fff7954c7-48vgr\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:27.813688 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.813691 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69b9013b-1695-4f0e-bc47-bbd8101610c8-oauth-serving-cert\") pod \"console-6fff7954c7-48vgr\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:27.813909 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.813713 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69b9013b-1695-4f0e-bc47-bbd8101610c8-console-config\") pod \"console-6fff7954c7-48vgr\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:27.813909 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.813730 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69b9013b-1695-4f0e-bc47-bbd8101610c8-trusted-ca-bundle\") pod \"console-6fff7954c7-48vgr\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:27.813909 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.813750 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69b9013b-1695-4f0e-bc47-bbd8101610c8-service-ca\") pod \"console-6fff7954c7-48vgr\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:27.813909 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.813806 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69b9013b-1695-4f0e-bc47-bbd8101610c8-console-serving-cert\") pod \"console-6fff7954c7-48vgr\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:27.813909 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.813847 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4blf\" (UniqueName: \"kubernetes.io/projected/69b9013b-1695-4f0e-bc47-bbd8101610c8-kube-api-access-s4blf\") pod \"console-6fff7954c7-48vgr\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:27.814633 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.814606 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69b9013b-1695-4f0e-bc47-bbd8101610c8-console-config\") pod \"console-6fff7954c7-48vgr\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:27.814755 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.814669 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69b9013b-1695-4f0e-bc47-bbd8101610c8-oauth-serving-cert\") pod \"console-6fff7954c7-48vgr\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:27.814755 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.814683 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69b9013b-1695-4f0e-bc47-bbd8101610c8-trusted-ca-bundle\") pod \"console-6fff7954c7-48vgr\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:27.814755 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.814742 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69b9013b-1695-4f0e-bc47-bbd8101610c8-service-ca\") pod \"console-6fff7954c7-48vgr\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:27.816309 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.816288 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69b9013b-1695-4f0e-bc47-bbd8101610c8-console-oauth-config\") pod \"console-6fff7954c7-48vgr\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:27.816387 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.816328 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69b9013b-1695-4f0e-bc47-bbd8101610c8-console-serving-cert\") pod \"console-6fff7954c7-48vgr\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:27.824655 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.824630 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4blf\" (UniqueName: \"kubernetes.io/projected/69b9013b-1695-4f0e-bc47-bbd8101610c8-kube-api-access-s4blf\") pod \"console-6fff7954c7-48vgr\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:27.896501 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:27.896407 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:28.034959 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:28.034919 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fff7954c7-48vgr"] Jul 23 13:11:28.037885 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:11:28.037858 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69b9013b_1695_4f0e_bc47_bbd8101610c8.slice/crio-5064daeea0fcedbd8d505201302ff1f7dd23267ad361aa5ea9520a5d6831667e WatchSource:0}: Error finding container 5064daeea0fcedbd8d505201302ff1f7dd23267ad361aa5ea9520a5d6831667e: Status 404 returned error can't find the container with id 5064daeea0fcedbd8d505201302ff1f7dd23267ad361aa5ea9520a5d6831667e Jul 23 13:11:28.491555 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:28.491512 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fff7954c7-48vgr" event={"ID":"69b9013b-1695-4f0e-bc47-bbd8101610c8","Type":"ContainerStarted","Data":"71d630d2795d22c092a524d2e117fbeb0bbef756213c2d19cafcbee047aa312d"} Jul 23 13:11:28.491728 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:28.491568 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fff7954c7-48vgr" event={"ID":"69b9013b-1695-4f0e-bc47-bbd8101610c8","Type":"ContainerStarted","Data":"5064daeea0fcedbd8d505201302ff1f7dd23267ad361aa5ea9520a5d6831667e"} Jul 23 13:11:28.511860 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:28.511805 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6fff7954c7-48vgr" podStartSLOduration=1.511788683 podStartE2EDuration="1.511788683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:11:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:11:28.510543721 +0000 UTC m=+225.749819698" watchObservedRunningTime="2026-07-23 13:11:28.511788683 +0000 UTC m=+225.751064659" Jul 23 13:11:37.896733 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:37.896694 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:37.897150 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:37.896753 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:37.901118 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:37.901101 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:38.530459 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:38.530430 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:11:38.581826 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:11:38.581788 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75bf6c86b4-dbrmr"] Jul 23 13:12:03.602895 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:03.602806 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-75bf6c86b4-dbrmr" podUID="6e1d9807-c931-4bc5-9127-3cf68e31b692" containerName="console" containerID="cri-o://acdddb5cd8cdd2dd49ef6f5e2163b3f786bc9b30d72e235e7b52b04989358026" gracePeriod=15 Jul 23 13:12:03.840918 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:03.840894 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:12:03.914966 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:03.914880 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e1d9807-c931-4bc5-9127-3cf68e31b692-console-serving-cert\") pod \"6e1d9807-c931-4bc5-9127-3cf68e31b692\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " Jul 23 13:12:03.914966 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:03.914942 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vjfs\" (UniqueName: \"kubernetes.io/projected/6e1d9807-c931-4bc5-9127-3cf68e31b692-kube-api-access-8vjfs\") pod \"6e1d9807-c931-4bc5-9127-3cf68e31b692\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " Jul 23 13:12:03.915166 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:03.914984 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e1d9807-c931-4bc5-9127-3cf68e31b692-trusted-ca-bundle\") pod \"6e1d9807-c931-4bc5-9127-3cf68e31b692\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " Jul 23 13:12:03.915166 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:03.915004 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e1d9807-c931-4bc5-9127-3cf68e31b692-oauth-serving-cert\") pod \"6e1d9807-c931-4bc5-9127-3cf68e31b692\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " Jul 23 13:12:03.915166 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:03.915027 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e1d9807-c931-4bc5-9127-3cf68e31b692-console-config\") pod \"6e1d9807-c931-4bc5-9127-3cf68e31b692\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " Jul 23 13:12:03.915166 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:03.915074 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e1d9807-c931-4bc5-9127-3cf68e31b692-console-oauth-config\") pod \"6e1d9807-c931-4bc5-9127-3cf68e31b692\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " Jul 23 13:12:03.915166 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:03.915109 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e1d9807-c931-4bc5-9127-3cf68e31b692-service-ca\") pod \"6e1d9807-c931-4bc5-9127-3cf68e31b692\" (UID: \"6e1d9807-c931-4bc5-9127-3cf68e31b692\") " Jul 23 13:12:03.915561 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:03.915534 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e1d9807-c931-4bc5-9127-3cf68e31b692-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6e1d9807-c931-4bc5-9127-3cf68e31b692" (UID: "6e1d9807-c931-4bc5-9127-3cf68e31b692"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:12:03.915561 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:03.915548 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e1d9807-c931-4bc5-9127-3cf68e31b692-console-config" (OuterVolumeSpecName: "console-config") pod "6e1d9807-c931-4bc5-9127-3cf68e31b692" (UID: "6e1d9807-c931-4bc5-9127-3cf68e31b692"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:12:03.915674 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:03.915548 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e1d9807-c931-4bc5-9127-3cf68e31b692-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6e1d9807-c931-4bc5-9127-3cf68e31b692" (UID: "6e1d9807-c931-4bc5-9127-3cf68e31b692"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:12:03.915674 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:03.915601 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e1d9807-c931-4bc5-9127-3cf68e31b692-service-ca" (OuterVolumeSpecName: "service-ca") pod "6e1d9807-c931-4bc5-9127-3cf68e31b692" (UID: "6e1d9807-c931-4bc5-9127-3cf68e31b692"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:12:03.917149 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:03.917126 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e1d9807-c931-4bc5-9127-3cf68e31b692-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6e1d9807-c931-4bc5-9127-3cf68e31b692" (UID: "6e1d9807-c931-4bc5-9127-3cf68e31b692"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 13:12:03.917849 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:03.917828 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e1d9807-c931-4bc5-9127-3cf68e31b692-kube-api-access-8vjfs" (OuterVolumeSpecName: "kube-api-access-8vjfs") pod "6e1d9807-c931-4bc5-9127-3cf68e31b692" (UID: "6e1d9807-c931-4bc5-9127-3cf68e31b692"). InnerVolumeSpecName "kube-api-access-8vjfs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:12:03.917849 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:03.917827 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e1d9807-c931-4bc5-9127-3cf68e31b692-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6e1d9807-c931-4bc5-9127-3cf68e31b692" (UID: "6e1d9807-c931-4bc5-9127-3cf68e31b692"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 13:12:04.016561 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:04.016515 2531 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e1d9807-c931-4bc5-9127-3cf68e31b692-console-serving-cert\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:12:04.016561 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:04.016555 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vjfs\" (UniqueName: \"kubernetes.io/projected/6e1d9807-c931-4bc5-9127-3cf68e31b692-kube-api-access-8vjfs\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:12:04.016561 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:04.016565 2531 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e1d9807-c931-4bc5-9127-3cf68e31b692-trusted-ca-bundle\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:12:04.016561 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:04.016574 2531 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e1d9807-c931-4bc5-9127-3cf68e31b692-oauth-serving-cert\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:12:04.016821 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:04.016582 2531 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e1d9807-c931-4bc5-9127-3cf68e31b692-console-config\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:12:04.016821 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:04.016590 2531 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e1d9807-c931-4bc5-9127-3cf68e31b692-console-oauth-config\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:12:04.016821 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:04.016598 2531 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e1d9807-c931-4bc5-9127-3cf68e31b692-service-ca\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:12:04.623495 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:04.623463 2531 generic.go:356] "Generic (PLEG): container finished" podID="6e1d9807-c931-4bc5-9127-3cf68e31b692" containerID="acdddb5cd8cdd2dd49ef6f5e2163b3f786bc9b30d72e235e7b52b04989358026" exitCode=0 Jul 23 13:12:04.623976 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:04.623520 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75bf6c86b4-dbrmr" Jul 23 13:12:04.623976 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:04.623540 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75bf6c86b4-dbrmr" event={"ID":"6e1d9807-c931-4bc5-9127-3cf68e31b692","Type":"ContainerDied","Data":"acdddb5cd8cdd2dd49ef6f5e2163b3f786bc9b30d72e235e7b52b04989358026"} Jul 23 13:12:04.623976 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:04.623580 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75bf6c86b4-dbrmr" event={"ID":"6e1d9807-c931-4bc5-9127-3cf68e31b692","Type":"ContainerDied","Data":"131f91634be8206c7a6fe11cb08662ba660f19682b333145e8cab0fda054a54e"} Jul 23 13:12:04.623976 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:04.623600 2531 scope.go:117] "RemoveContainer" containerID="acdddb5cd8cdd2dd49ef6f5e2163b3f786bc9b30d72e235e7b52b04989358026" Jul 23 13:12:04.631828 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:04.631813 2531 scope.go:117] "RemoveContainer" containerID="acdddb5cd8cdd2dd49ef6f5e2163b3f786bc9b30d72e235e7b52b04989358026" Jul 23 13:12:04.632074 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:12:04.632058 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acdddb5cd8cdd2dd49ef6f5e2163b3f786bc9b30d72e235e7b52b04989358026\": container with ID starting with acdddb5cd8cdd2dd49ef6f5e2163b3f786bc9b30d72e235e7b52b04989358026 not found: ID does not exist" containerID="acdddb5cd8cdd2dd49ef6f5e2163b3f786bc9b30d72e235e7b52b04989358026" Jul 23 13:12:04.632125 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:04.632083 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acdddb5cd8cdd2dd49ef6f5e2163b3f786bc9b30d72e235e7b52b04989358026"} err="failed to get container status \"acdddb5cd8cdd2dd49ef6f5e2163b3f786bc9b30d72e235e7b52b04989358026\": rpc error: code = NotFound desc = could not find container \"acdddb5cd8cdd2dd49ef6f5e2163b3f786bc9b30d72e235e7b52b04989358026\": container with ID starting with acdddb5cd8cdd2dd49ef6f5e2163b3f786bc9b30d72e235e7b52b04989358026 not found: ID does not exist" Jul 23 13:12:04.643068 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:04.643046 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75bf6c86b4-dbrmr"] Jul 23 13:12:04.647297 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:04.647278 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75bf6c86b4-dbrmr"] Jul 23 13:12:05.330920 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:05.330890 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e1d9807-c931-4bc5-9127-3cf68e31b692" path="/var/lib/kubelet/pods/6e1d9807-c931-4bc5-9127-3cf68e31b692/volumes" Jul 23 13:12:28.236478 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:28.236433 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n"] Jul 23 13:12:28.236916 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:28.236738 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e1d9807-c931-4bc5-9127-3cf68e31b692" containerName="console" Jul 23 13:12:28.236916 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:28.236749 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e1d9807-c931-4bc5-9127-3cf68e31b692" containerName="console" Jul 23 13:12:28.236916 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:28.236789 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e1d9807-c931-4bc5-9127-3cf68e31b692" containerName="console" Jul 23 13:12:28.241457 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:28.241437 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n" Jul 23 13:12:28.244772 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:28.244755 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xzjlp\"" Jul 23 13:12:28.244883 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:28.244755 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 23 13:12:28.245573 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:28.245557 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 23 13:12:28.249881 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:28.249859 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n"] Jul 23 13:12:28.323312 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:28.323281 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfdnh\" (UniqueName: \"kubernetes.io/projected/76674d9a-bf4f-4a49-ad3e-59a37a9e809e-kube-api-access-vfdnh\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n\" (UID: \"76674d9a-bf4f-4a49-ad3e-59a37a9e809e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n" Jul 23 13:12:28.323482 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:28.323339 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76674d9a-bf4f-4a49-ad3e-59a37a9e809e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n\" (UID: \"76674d9a-bf4f-4a49-ad3e-59a37a9e809e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n" Jul 23 13:12:28.323482 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:28.323391 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76674d9a-bf4f-4a49-ad3e-59a37a9e809e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n\" (UID: \"76674d9a-bf4f-4a49-ad3e-59a37a9e809e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n" Jul 23 13:12:28.424377 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:28.424323 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfdnh\" (UniqueName: \"kubernetes.io/projected/76674d9a-bf4f-4a49-ad3e-59a37a9e809e-kube-api-access-vfdnh\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n\" (UID: \"76674d9a-bf4f-4a49-ad3e-59a37a9e809e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n" Jul 23 13:12:28.424548 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:28.424412 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76674d9a-bf4f-4a49-ad3e-59a37a9e809e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n\" (UID: \"76674d9a-bf4f-4a49-ad3e-59a37a9e809e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n" Jul 23 13:12:28.424548 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:28.424444 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76674d9a-bf4f-4a49-ad3e-59a37a9e809e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n\" (UID: \"76674d9a-bf4f-4a49-ad3e-59a37a9e809e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n" Jul 23 13:12:28.424811 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:28.424793 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76674d9a-bf4f-4a49-ad3e-59a37a9e809e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n\" (UID: \"76674d9a-bf4f-4a49-ad3e-59a37a9e809e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n" Jul 23 13:12:28.424846 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:28.424820 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76674d9a-bf4f-4a49-ad3e-59a37a9e809e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n\" (UID: \"76674d9a-bf4f-4a49-ad3e-59a37a9e809e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n" Jul 23 13:12:28.434140 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:28.434112 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfdnh\" (UniqueName: \"kubernetes.io/projected/76674d9a-bf4f-4a49-ad3e-59a37a9e809e-kube-api-access-vfdnh\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n\" (UID: \"76674d9a-bf4f-4a49-ad3e-59a37a9e809e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n" Jul 23 13:12:28.550919 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:28.550836 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n" Jul 23 13:12:28.686689 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:28.686519 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n"] Jul 23 13:12:28.688917 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:12:28.688880 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76674d9a_bf4f_4a49_ad3e_59a37a9e809e.slice/crio-276b6500327572ca8a2c8c78eb16040c90a1daca7ecb873c510df5b5ed43c6c4 WatchSource:0}: Error finding container 276b6500327572ca8a2c8c78eb16040c90a1daca7ecb873c510df5b5ed43c6c4: Status 404 returned error can't find the container with id 276b6500327572ca8a2c8c78eb16040c90a1daca7ecb873c510df5b5ed43c6c4 Jul 23 13:12:28.710198 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:28.710170 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n" event={"ID":"76674d9a-bf4f-4a49-ad3e-59a37a9e809e","Type":"ContainerStarted","Data":"276b6500327572ca8a2c8c78eb16040c90a1daca7ecb873c510df5b5ed43c6c4"} Jul 23 13:12:36.744595 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:36.744558 2531 generic.go:356] "Generic (PLEG): container finished" podID="76674d9a-bf4f-4a49-ad3e-59a37a9e809e" containerID="a6e4b249da78edc6b4ec1806f3a4228cdce3751b953ff2cdfa94faef920ce1c9" exitCode=0 Jul 23 13:12:36.745064 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:36.744650 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n" event={"ID":"76674d9a-bf4f-4a49-ad3e-59a37a9e809e","Type":"ContainerDied","Data":"a6e4b249da78edc6b4ec1806f3a4228cdce3751b953ff2cdfa94faef920ce1c9"} Jul 23 13:12:38.756049 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:38.756018 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n" event={"ID":"76674d9a-bf4f-4a49-ad3e-59a37a9e809e","Type":"ContainerStarted","Data":"2b6b6350595dc35d750d4db2d8007f3d3c1cf79a93b00b86d262db6e7ae610aa"} Jul 23 13:12:39.761555 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:39.761523 2531 generic.go:356] "Generic (PLEG): container finished" podID="76674d9a-bf4f-4a49-ad3e-59a37a9e809e" containerID="2b6b6350595dc35d750d4db2d8007f3d3c1cf79a93b00b86d262db6e7ae610aa" exitCode=0 Jul 23 13:12:39.761930 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:39.761596 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n" event={"ID":"76674d9a-bf4f-4a49-ad3e-59a37a9e809e","Type":"ContainerDied","Data":"2b6b6350595dc35d750d4db2d8007f3d3c1cf79a93b00b86d262db6e7ae610aa"} Jul 23 13:12:43.170315 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:43.170288 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6m8j_a0a5f6bc-bdb7-46ae-8cb9-79edd5179817/ovn-acl-logging/0.log" Jul 23 13:12:43.173970 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:43.173946 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6m8j_a0a5f6bc-bdb7-46ae-8cb9-79edd5179817/ovn-acl-logging/0.log" Jul 23 13:12:43.176909 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:43.176890 2531 kubelet.go:1648] "Image garbage collection succeeded" Jul 23 13:12:46.793168 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:46.793134 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n" event={"ID":"76674d9a-bf4f-4a49-ad3e-59a37a9e809e","Type":"ContainerStarted","Data":"ace2696525b81446a3bc3a2b87eb19bd958fc4de80f7d5ace0f53343241c8640"} Jul 23 13:12:46.813748 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:46.813690 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n" podStartSLOduration=2.675533573 podStartE2EDuration="18.813674932s" totalImagesPullingTime="16.138141359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:12:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:12:46.812221447 +0000 UTC m=+304.051497426" watchObservedRunningTime="2026-07-23 13:12:46.813674932 +0000 UTC m=+304.052950942" Jul 23 13:12:47.798802 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:47.798764 2531 generic.go:356] "Generic (PLEG): container finished" podID="76674d9a-bf4f-4a49-ad3e-59a37a9e809e" containerID="ace2696525b81446a3bc3a2b87eb19bd958fc4de80f7d5ace0f53343241c8640" exitCode=0 Jul 23 13:12:47.799165 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:47.798829 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n" event={"ID":"76674d9a-bf4f-4a49-ad3e-59a37a9e809e","Type":"ContainerDied","Data":"ace2696525b81446a3bc3a2b87eb19bd958fc4de80f7d5ace0f53343241c8640"} Jul 23 13:12:48.927302 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:48.927277 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n" Jul 23 13:12:48.999318 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:48.999286 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76674d9a-bf4f-4a49-ad3e-59a37a9e809e-bundle\") pod \"76674d9a-bf4f-4a49-ad3e-59a37a9e809e\" (UID: \"76674d9a-bf4f-4a49-ad3e-59a37a9e809e\") " Jul 23 13:12:48.999318 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:48.999323 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76674d9a-bf4f-4a49-ad3e-59a37a9e809e-util\") pod \"76674d9a-bf4f-4a49-ad3e-59a37a9e809e\" (UID: \"76674d9a-bf4f-4a49-ad3e-59a37a9e809e\") " Jul 23 13:12:48.999525 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:48.999415 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfdnh\" (UniqueName: \"kubernetes.io/projected/76674d9a-bf4f-4a49-ad3e-59a37a9e809e-kube-api-access-vfdnh\") pod \"76674d9a-bf4f-4a49-ad3e-59a37a9e809e\" (UID: \"76674d9a-bf4f-4a49-ad3e-59a37a9e809e\") " Jul 23 13:12:48.999965 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:48.999933 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76674d9a-bf4f-4a49-ad3e-59a37a9e809e-bundle" (OuterVolumeSpecName: "bundle") pod "76674d9a-bf4f-4a49-ad3e-59a37a9e809e" (UID: "76674d9a-bf4f-4a49-ad3e-59a37a9e809e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:12:49.001759 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:49.001731 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76674d9a-bf4f-4a49-ad3e-59a37a9e809e-kube-api-access-vfdnh" (OuterVolumeSpecName: "kube-api-access-vfdnh") pod "76674d9a-bf4f-4a49-ad3e-59a37a9e809e" (UID: "76674d9a-bf4f-4a49-ad3e-59a37a9e809e"). InnerVolumeSpecName "kube-api-access-vfdnh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:12:49.003935 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:49.003910 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76674d9a-bf4f-4a49-ad3e-59a37a9e809e-util" (OuterVolumeSpecName: "util") pod "76674d9a-bf4f-4a49-ad3e-59a37a9e809e" (UID: "76674d9a-bf4f-4a49-ad3e-59a37a9e809e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:12:49.100572 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:49.100546 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfdnh\" (UniqueName: \"kubernetes.io/projected/76674d9a-bf4f-4a49-ad3e-59a37a9e809e-kube-api-access-vfdnh\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:12:49.100572 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:49.100572 2531 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76674d9a-bf4f-4a49-ad3e-59a37a9e809e-bundle\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:12:49.100740 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:49.100581 2531 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76674d9a-bf4f-4a49-ad3e-59a37a9e809e-util\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:12:49.808305 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:49.808267 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n" event={"ID":"76674d9a-bf4f-4a49-ad3e-59a37a9e809e","Type":"ContainerDied","Data":"276b6500327572ca8a2c8c78eb16040c90a1daca7ecb873c510df5b5ed43c6c4"} Jul 23 13:12:49.808305 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:49.808280 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mbt9n" Jul 23 13:12:49.808552 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:49.808315 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="276b6500327572ca8a2c8c78eb16040c90a1daca7ecb873c510df5b5ed43c6c4" Jul 23 13:12:56.854521 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:56.854489 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k27zq"] Jul 23 13:12:56.854900 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:56.854770 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76674d9a-bf4f-4a49-ad3e-59a37a9e809e" containerName="pull" Jul 23 13:12:56.854900 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:56.854779 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="76674d9a-bf4f-4a49-ad3e-59a37a9e809e" containerName="pull" Jul 23 13:12:56.854900 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:56.854791 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76674d9a-bf4f-4a49-ad3e-59a37a9e809e" containerName="util" Jul 23 13:12:56.854900 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:56.854797 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="76674d9a-bf4f-4a49-ad3e-59a37a9e809e" containerName="util" Jul 23 13:12:56.854900 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:56.854805 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76674d9a-bf4f-4a49-ad3e-59a37a9e809e" containerName="extract" Jul 23 13:12:56.854900 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:56.854811 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="76674d9a-bf4f-4a49-ad3e-59a37a9e809e" containerName="extract" Jul 23 13:12:56.854900 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:56.854850 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="76674d9a-bf4f-4a49-ad3e-59a37a9e809e" containerName="extract" Jul 23 13:12:56.881931 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:56.881905 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k27zq" Jul 23 13:12:56.886307 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:56.886285 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 23 13:12:56.888561 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:56.888540 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 23 13:12:56.888655 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:56.888545 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-q86jc\"" Jul 23 13:12:56.903777 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:56.903753 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k27zq"] Jul 23 13:12:56.964404 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:56.964373 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcqtv\" (UniqueName: \"kubernetes.io/projected/7136358b-5679-467b-878b-e14b4092d5ff-kube-api-access-hcqtv\") pod \"cert-manager-operator-controller-manager-55c444bf86-k27zq\" (UID: \"7136358b-5679-467b-878b-e14b4092d5ff\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k27zq" Jul 23 13:12:56.964594 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:56.964449 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7136358b-5679-467b-878b-e14b4092d5ff-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-k27zq\" (UID: \"7136358b-5679-467b-878b-e14b4092d5ff\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k27zq" Jul 23 13:12:57.065605 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:57.065573 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7136358b-5679-467b-878b-e14b4092d5ff-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-k27zq\" (UID: \"7136358b-5679-467b-878b-e14b4092d5ff\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k27zq" Jul 23 13:12:57.065764 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:57.065618 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcqtv\" (UniqueName: \"kubernetes.io/projected/7136358b-5679-467b-878b-e14b4092d5ff-kube-api-access-hcqtv\") pod \"cert-manager-operator-controller-manager-55c444bf86-k27zq\" (UID: \"7136358b-5679-467b-878b-e14b4092d5ff\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k27zq" Jul 23 13:12:57.065976 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:57.065959 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7136358b-5679-467b-878b-e14b4092d5ff-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-k27zq\" (UID: \"7136358b-5679-467b-878b-e14b4092d5ff\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k27zq" Jul 23 13:12:57.084208 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:57.084177 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcqtv\" (UniqueName: \"kubernetes.io/projected/7136358b-5679-467b-878b-e14b4092d5ff-kube-api-access-hcqtv\") pod \"cert-manager-operator-controller-manager-55c444bf86-k27zq\" (UID: \"7136358b-5679-467b-878b-e14b4092d5ff\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k27zq" Jul 23 13:12:57.190979 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:57.190902 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k27zq" Jul 23 13:12:57.363092 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:57.363059 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k27zq"] Jul 23 13:12:57.366620 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:12:57.366594 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7136358b_5679_467b_878b_e14b4092d5ff.slice/crio-0054dbe7800ecc3e532513712052a7afd73dc0b9f8299ccf139a650b8bb309bf WatchSource:0}: Error finding container 0054dbe7800ecc3e532513712052a7afd73dc0b9f8299ccf139a650b8bb309bf: Status 404 returned error can't find the container with id 0054dbe7800ecc3e532513712052a7afd73dc0b9f8299ccf139a650b8bb309bf Jul 23 13:12:57.369115 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:57.369101 2531 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 13:12:57.838521 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:57.838486 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k27zq" event={"ID":"7136358b-5679-467b-878b-e14b4092d5ff","Type":"ContainerStarted","Data":"0054dbe7800ecc3e532513712052a7afd73dc0b9f8299ccf139a650b8bb309bf"} Jul 23 13:12:59.849658 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:59.849618 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k27zq" event={"ID":"7136358b-5679-467b-878b-e14b4092d5ff","Type":"ContainerStarted","Data":"86e6dc255e9b016cd348b91fb4a5082197729c30792ad3bfbda1fe7b61427b7f"} Jul 23 13:12:59.906078 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:12:59.906022 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k27zq" podStartSLOduration=1.6082616079999998 podStartE2EDuration="3.906006208s" totalImagesPullingTime="2.2977446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:12:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:12:59.90424726 +0000 UTC m=+317.143523235" watchObservedRunningTime="2026-07-23 13:12:59.906006208 +0000 UTC m=+317.145282183" Jul 23 13:13:01.053860 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:01.053823 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4"] Jul 23 13:13:01.056837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:01.056821 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4" Jul 23 13:13:01.064216 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:01.064192 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 23 13:13:01.064896 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:01.064878 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 23 13:13:01.065239 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:01.065225 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xzjlp\"" Jul 23 13:13:01.096930 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:01.096904 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebb73171-995b-4591-be0e-89c3760866f1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4\" (UID: \"ebb73171-995b-4591-be0e-89c3760866f1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4" Jul 23 13:13:01.097038 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:01.096945 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebb73171-995b-4591-be0e-89c3760866f1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4\" (UID: \"ebb73171-995b-4591-be0e-89c3760866f1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4" Jul 23 13:13:01.097038 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:01.096976 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pphh\" (UniqueName: \"kubernetes.io/projected/ebb73171-995b-4591-be0e-89c3760866f1-kube-api-access-9pphh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4\" (UID: \"ebb73171-995b-4591-be0e-89c3760866f1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4" Jul 23 13:13:01.123135 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:01.123108 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4"] Jul 23 13:13:01.197586 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:01.197553 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebb73171-995b-4591-be0e-89c3760866f1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4\" (UID: \"ebb73171-995b-4591-be0e-89c3760866f1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4" Jul 23 13:13:01.197719 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:01.197599 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebb73171-995b-4591-be0e-89c3760866f1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4\" (UID: \"ebb73171-995b-4591-be0e-89c3760866f1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4" Jul 23 13:13:01.197719 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:01.197629 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pphh\" (UniqueName: \"kubernetes.io/projected/ebb73171-995b-4591-be0e-89c3760866f1-kube-api-access-9pphh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4\" (UID: \"ebb73171-995b-4591-be0e-89c3760866f1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4" Jul 23 13:13:01.197956 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:01.197940 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebb73171-995b-4591-be0e-89c3760866f1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4\" (UID: \"ebb73171-995b-4591-be0e-89c3760866f1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4" Jul 23 13:13:01.197995 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:01.197972 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebb73171-995b-4591-be0e-89c3760866f1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4\" (UID: \"ebb73171-995b-4591-be0e-89c3760866f1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4" Jul 23 13:13:01.228511 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:01.228480 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pphh\" (UniqueName: \"kubernetes.io/projected/ebb73171-995b-4591-be0e-89c3760866f1-kube-api-access-9pphh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4\" (UID: \"ebb73171-995b-4591-be0e-89c3760866f1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4" Jul 23 13:13:01.365730 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:01.365699 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4" Jul 23 13:13:01.527248 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:01.527221 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4"] Jul 23 13:13:01.528712 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:13:01.528685 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebb73171_995b_4591_be0e_89c3760866f1.slice/crio-51c6c994dd313dcfa7dea86043881416c3448633037bab11b2992189788f7c96 WatchSource:0}: Error finding container 51c6c994dd313dcfa7dea86043881416c3448633037bab11b2992189788f7c96: Status 404 returned error can't find the container with id 51c6c994dd313dcfa7dea86043881416c3448633037bab11b2992189788f7c96 Jul 23 13:13:01.859812 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:01.859779 2531 generic.go:356] "Generic (PLEG): container finished" podID="ebb73171-995b-4591-be0e-89c3760866f1" containerID="336c6602cbe665571c16e4d7e9ab9e662ce2fc1f55d6a884a579b191dd93d6f7" exitCode=0 Jul 23 13:13:01.859976 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:01.859868 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4" event={"ID":"ebb73171-995b-4591-be0e-89c3760866f1","Type":"ContainerDied","Data":"336c6602cbe665571c16e4d7e9ab9e662ce2fc1f55d6a884a579b191dd93d6f7"} Jul 23 13:13:01.859976 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:01.859917 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4" event={"ID":"ebb73171-995b-4591-be0e-89c3760866f1","Type":"ContainerStarted","Data":"51c6c994dd313dcfa7dea86043881416c3448633037bab11b2992189788f7c96"} Jul 23 13:13:03.929049 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:03.929012 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-k2s79"] Jul 23 13:13:03.935895 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:03.935873 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-k2s79" Jul 23 13:13:03.940228 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:03.940204 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 23 13:13:03.940843 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:03.940783 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-4t6pz\"" Jul 23 13:13:03.940985 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:03.940891 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 23 13:13:03.959573 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:03.959541 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-k2s79"] Jul 23 13:13:04.022343 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:04.022303 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q4nb\" (UniqueName: \"kubernetes.io/projected/b4fdc520-9278-4370-afd0-fe51b5951564-kube-api-access-4q4nb\") pod \"cert-manager-cainjector-559cdd8c87-k2s79\" (UID: \"b4fdc520-9278-4370-afd0-fe51b5951564\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-k2s79" Jul 23 13:13:04.022532 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:04.022436 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b4fdc520-9278-4370-afd0-fe51b5951564-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-k2s79\" (UID: \"b4fdc520-9278-4370-afd0-fe51b5951564\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-k2s79" Jul 23 13:13:04.123725 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:04.123683 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b4fdc520-9278-4370-afd0-fe51b5951564-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-k2s79\" (UID: \"b4fdc520-9278-4370-afd0-fe51b5951564\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-k2s79" Jul 23 13:13:04.123913 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:04.123746 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q4nb\" (UniqueName: \"kubernetes.io/projected/b4fdc520-9278-4370-afd0-fe51b5951564-kube-api-access-4q4nb\") pod \"cert-manager-cainjector-559cdd8c87-k2s79\" (UID: \"b4fdc520-9278-4370-afd0-fe51b5951564\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-k2s79" Jul 23 13:13:04.157283 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:04.157249 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b4fdc520-9278-4370-afd0-fe51b5951564-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-k2s79\" (UID: \"b4fdc520-9278-4370-afd0-fe51b5951564\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-k2s79" Jul 23 13:13:04.157465 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:04.157442 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q4nb\" (UniqueName: \"kubernetes.io/projected/b4fdc520-9278-4370-afd0-fe51b5951564-kube-api-access-4q4nb\") pod \"cert-manager-cainjector-559cdd8c87-k2s79\" (UID: \"b4fdc520-9278-4370-afd0-fe51b5951564\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-k2s79" Jul 23 13:13:04.259475 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:04.259403 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-k2s79" Jul 23 13:13:04.432319 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:04.432291 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-k2s79"] Jul 23 13:13:04.454132 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:13:04.454102 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4fdc520_9278_4370_afd0_fe51b5951564.slice/crio-e35e232db1f623ae1537fa1dd331e28a6f8e629fa87d04c55887c0a618b643a6 WatchSource:0}: Error finding container e35e232db1f623ae1537fa1dd331e28a6f8e629fa87d04c55887c0a618b643a6: Status 404 returned error can't find the container with id e35e232db1f623ae1537fa1dd331e28a6f8e629fa87d04c55887c0a618b643a6 Jul 23 13:13:04.873669 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:04.873635 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-k2s79" event={"ID":"b4fdc520-9278-4370-afd0-fe51b5951564","Type":"ContainerStarted","Data":"e35e232db1f623ae1537fa1dd331e28a6f8e629fa87d04c55887c0a618b643a6"} Jul 23 13:13:04.875161 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:04.875137 2531 generic.go:356] "Generic (PLEG): container finished" podID="ebb73171-995b-4591-be0e-89c3760866f1" containerID="4ac0dcbbe85b0b265f9a46b2c72027b952a744ebf7313283b172a55071f7e7de" exitCode=0 Jul 23 13:13:04.875340 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:04.875208 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4" event={"ID":"ebb73171-995b-4591-be0e-89c3760866f1","Type":"ContainerDied","Data":"4ac0dcbbe85b0b265f9a46b2c72027b952a744ebf7313283b172a55071f7e7de"} Jul 23 13:13:05.883279 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:05.883244 2531 generic.go:356] "Generic (PLEG): container finished" podID="ebb73171-995b-4591-be0e-89c3760866f1" containerID="98bd0a9d865bf2c1ab3ebcbdf4ae99877d241223d5aee655a44c84ff3593fef7" exitCode=0 Jul 23 13:13:05.883756 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:05.883348 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4" event={"ID":"ebb73171-995b-4591-be0e-89c3760866f1","Type":"ContainerDied","Data":"98bd0a9d865bf2c1ab3ebcbdf4ae99877d241223d5aee655a44c84ff3593fef7"} Jul 23 13:13:07.409033 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:07.409011 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4" Jul 23 13:13:07.552790 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:07.552762 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebb73171-995b-4591-be0e-89c3760866f1-util\") pod \"ebb73171-995b-4591-be0e-89c3760866f1\" (UID: \"ebb73171-995b-4591-be0e-89c3760866f1\") " Jul 23 13:13:07.552984 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:07.552799 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pphh\" (UniqueName: \"kubernetes.io/projected/ebb73171-995b-4591-be0e-89c3760866f1-kube-api-access-9pphh\") pod \"ebb73171-995b-4591-be0e-89c3760866f1\" (UID: \"ebb73171-995b-4591-be0e-89c3760866f1\") " Jul 23 13:13:07.552984 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:07.552852 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebb73171-995b-4591-be0e-89c3760866f1-bundle\") pod \"ebb73171-995b-4591-be0e-89c3760866f1\" (UID: \"ebb73171-995b-4591-be0e-89c3760866f1\") " Jul 23 13:13:07.553212 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:07.553191 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebb73171-995b-4591-be0e-89c3760866f1-bundle" (OuterVolumeSpecName: "bundle") pod "ebb73171-995b-4591-be0e-89c3760866f1" (UID: "ebb73171-995b-4591-be0e-89c3760866f1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:13:07.555027 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:07.555003 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebb73171-995b-4591-be0e-89c3760866f1-kube-api-access-9pphh" (OuterVolumeSpecName: "kube-api-access-9pphh") pod "ebb73171-995b-4591-be0e-89c3760866f1" (UID: "ebb73171-995b-4591-be0e-89c3760866f1"). InnerVolumeSpecName "kube-api-access-9pphh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:13:07.556910 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:07.556882 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebb73171-995b-4591-be0e-89c3760866f1-util" (OuterVolumeSpecName: "util") pod "ebb73171-995b-4591-be0e-89c3760866f1" (UID: "ebb73171-995b-4591-be0e-89c3760866f1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:13:07.653467 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:07.653337 2531 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebb73171-995b-4591-be0e-89c3760866f1-util\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:13:07.653467 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:07.653384 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pphh\" (UniqueName: \"kubernetes.io/projected/ebb73171-995b-4591-be0e-89c3760866f1-kube-api-access-9pphh\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:13:07.653467 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:07.653398 2531 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebb73171-995b-4591-be0e-89c3760866f1-bundle\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:13:07.895494 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:07.895457 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-k2s79" event={"ID":"b4fdc520-9278-4370-afd0-fe51b5951564","Type":"ContainerStarted","Data":"0ddbb8f71e7a205402bad5b6c0151be7979e0045eea0f3be237b2e31f645506c"} Jul 23 13:13:07.897489 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:07.897462 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4" event={"ID":"ebb73171-995b-4591-be0e-89c3760866f1","Type":"ContainerDied","Data":"51c6c994dd313dcfa7dea86043881416c3448633037bab11b2992189788f7c96"} Jul 23 13:13:07.897618 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:07.897510 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51c6c994dd313dcfa7dea86043881416c3448633037bab11b2992189788f7c96" Jul 23 13:13:07.897618 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:07.897472 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kpj4" Jul 23 13:13:07.951047 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:07.950987 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-k2s79" podStartSLOduration=1.966761585 podStartE2EDuration="4.950965552s" totalImagesPullingTime="2.984203967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:13:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:13:07.950093906 +0000 UTC m=+325.189369883" watchObservedRunningTime="2026-07-23 13:13:07.950965552 +0000 UTC m=+325.190241529" Jul 23 13:13:12.322814 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:12.322779 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-vzsk4"] Jul 23 13:13:12.323184 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:12.323072 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebb73171-995b-4591-be0e-89c3760866f1" containerName="pull" Jul 23 13:13:12.323184 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:12.323084 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebb73171-995b-4591-be0e-89c3760866f1" containerName="pull" Jul 23 13:13:12.323184 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:12.323108 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebb73171-995b-4591-be0e-89c3760866f1" containerName="util" Jul 23 13:13:12.323184 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:12.323113 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebb73171-995b-4591-be0e-89c3760866f1" containerName="util" Jul 23 13:13:12.323184 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:12.323119 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebb73171-995b-4591-be0e-89c3760866f1" containerName="extract" Jul 23 13:13:12.323184 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:12.323124 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebb73171-995b-4591-be0e-89c3760866f1" containerName="extract" Jul 23 13:13:12.323184 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:12.323173 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebb73171-995b-4591-be0e-89c3760866f1" containerName="extract" Jul 23 13:13:12.325236 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:12.325220 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-vzsk4" Jul 23 13:13:12.328465 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:12.328443 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-qmzkk\"" Jul 23 13:13:12.342050 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:12.342026 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-vzsk4"] Jul 23 13:13:12.390169 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:12.390124 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/71e8e3a4-fe86-49ad-beab-9d3e180b9722-bound-sa-token\") pod \"cert-manager-55b4d954d5-vzsk4\" (UID: \"71e8e3a4-fe86-49ad-beab-9d3e180b9722\") " pod="cert-manager/cert-manager-55b4d954d5-vzsk4" Jul 23 13:13:12.390350 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:12.390236 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2c8f\" (UniqueName: \"kubernetes.io/projected/71e8e3a4-fe86-49ad-beab-9d3e180b9722-kube-api-access-z2c8f\") pod \"cert-manager-55b4d954d5-vzsk4\" (UID: \"71e8e3a4-fe86-49ad-beab-9d3e180b9722\") " pod="cert-manager/cert-manager-55b4d954d5-vzsk4" Jul 23 13:13:12.491510 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:12.491474 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/71e8e3a4-fe86-49ad-beab-9d3e180b9722-bound-sa-token\") pod \"cert-manager-55b4d954d5-vzsk4\" (UID: \"71e8e3a4-fe86-49ad-beab-9d3e180b9722\") " pod="cert-manager/cert-manager-55b4d954d5-vzsk4" Jul 23 13:13:12.491654 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:12.491536 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2c8f\" (UniqueName: \"kubernetes.io/projected/71e8e3a4-fe86-49ad-beab-9d3e180b9722-kube-api-access-z2c8f\") pod \"cert-manager-55b4d954d5-vzsk4\" (UID: \"71e8e3a4-fe86-49ad-beab-9d3e180b9722\") " pod="cert-manager/cert-manager-55b4d954d5-vzsk4" Jul 23 13:13:12.503639 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:12.503606 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/71e8e3a4-fe86-49ad-beab-9d3e180b9722-bound-sa-token\") pod \"cert-manager-55b4d954d5-vzsk4\" (UID: \"71e8e3a4-fe86-49ad-beab-9d3e180b9722\") " pod="cert-manager/cert-manager-55b4d954d5-vzsk4" Jul 23 13:13:12.505134 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:12.505109 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2c8f\" (UniqueName: \"kubernetes.io/projected/71e8e3a4-fe86-49ad-beab-9d3e180b9722-kube-api-access-z2c8f\") pod \"cert-manager-55b4d954d5-vzsk4\" (UID: \"71e8e3a4-fe86-49ad-beab-9d3e180b9722\") " pod="cert-manager/cert-manager-55b4d954d5-vzsk4" Jul 23 13:13:12.633855 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:12.633767 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-vzsk4" Jul 23 13:13:12.772936 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:12.772897 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-vzsk4"] Jul 23 13:13:12.776825 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:13:12.776797 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71e8e3a4_fe86_49ad_beab_9d3e180b9722.slice/crio-59e8e3670e8535492063e33e98901c9156972eacbcf04f0177136735df3c1e4d WatchSource:0}: Error finding container 59e8e3670e8535492063e33e98901c9156972eacbcf04f0177136735df3c1e4d: Status 404 returned error can't find the container with id 59e8e3670e8535492063e33e98901c9156972eacbcf04f0177136735df3c1e4d Jul 23 13:13:12.919213 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:12.919118 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-vzsk4" event={"ID":"71e8e3a4-fe86-49ad-beab-9d3e180b9722","Type":"ContainerStarted","Data":"a18a782f06b18602616787836897dd1f4f73dfd489478857d09a795398329d3c"} Jul 23 13:13:12.919213 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:12.919163 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-vzsk4" event={"ID":"71e8e3a4-fe86-49ad-beab-9d3e180b9722","Type":"ContainerStarted","Data":"59e8e3670e8535492063e33e98901c9156972eacbcf04f0177136735df3c1e4d"} Jul 23 13:13:12.940393 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:12.940324 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-vzsk4" podStartSLOduration=0.940309237 podStartE2EDuration="940.309237ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:13:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:13:12.938393901 +0000 UTC m=+330.177669917" watchObservedRunningTime="2026-07-23 13:13:12.940309237 +0000 UTC m=+330.179585212" Jul 23 13:13:21.409407 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:21.409373 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz"] Jul 23 13:13:21.412198 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:21.412182 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz" Jul 23 13:13:21.416284 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:21.416262 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 23 13:13:21.416411 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:21.416297 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 23 13:13:21.417205 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:21.417189 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xzjlp\"" Jul 23 13:13:21.425052 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:21.425029 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz"] Jul 23 13:13:21.569030 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:21.568992 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9f3163a-e8ec-4cd6-9a15-17c6310f3d76-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz\" (UID: \"e9f3163a-e8ec-4cd6-9a15-17c6310f3d76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz" Jul 23 13:13:21.569198 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:21.569045 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bghdm\" (UniqueName: \"kubernetes.io/projected/e9f3163a-e8ec-4cd6-9a15-17c6310f3d76-kube-api-access-bghdm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz\" (UID: \"e9f3163a-e8ec-4cd6-9a15-17c6310f3d76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz" Jul 23 13:13:21.569198 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:21.569089 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9f3163a-e8ec-4cd6-9a15-17c6310f3d76-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz\" (UID: \"e9f3163a-e8ec-4cd6-9a15-17c6310f3d76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz" Jul 23 13:13:21.669619 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:21.669539 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9f3163a-e8ec-4cd6-9a15-17c6310f3d76-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz\" (UID: \"e9f3163a-e8ec-4cd6-9a15-17c6310f3d76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz" Jul 23 13:13:21.669619 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:21.669598 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bghdm\" (UniqueName: \"kubernetes.io/projected/e9f3163a-e8ec-4cd6-9a15-17c6310f3d76-kube-api-access-bghdm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz\" (UID: \"e9f3163a-e8ec-4cd6-9a15-17c6310f3d76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz" Jul 23 13:13:21.669801 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:21.669635 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9f3163a-e8ec-4cd6-9a15-17c6310f3d76-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz\" (UID: \"e9f3163a-e8ec-4cd6-9a15-17c6310f3d76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz" Jul 23 13:13:21.670000 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:21.669979 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9f3163a-e8ec-4cd6-9a15-17c6310f3d76-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz\" (UID: \"e9f3163a-e8ec-4cd6-9a15-17c6310f3d76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz" Jul 23 13:13:21.670039 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:21.669996 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9f3163a-e8ec-4cd6-9a15-17c6310f3d76-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz\" (UID: \"e9f3163a-e8ec-4cd6-9a15-17c6310f3d76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz" Jul 23 13:13:21.677867 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:21.677847 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bghdm\" (UniqueName: \"kubernetes.io/projected/e9f3163a-e8ec-4cd6-9a15-17c6310f3d76-kube-api-access-bghdm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz\" (UID: \"e9f3163a-e8ec-4cd6-9a15-17c6310f3d76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz" Jul 23 13:13:21.721498 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:21.721456 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz" Jul 23 13:13:21.856659 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:21.856627 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz"] Jul 23 13:13:21.859870 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:13:21.859843 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9f3163a_e8ec_4cd6_9a15_17c6310f3d76.slice/crio-870c223e98ac7230aa82ae0ccc2fb53dcec6af9e52070cf838368c721d0a9049 WatchSource:0}: Error finding container 870c223e98ac7230aa82ae0ccc2fb53dcec6af9e52070cf838368c721d0a9049: Status 404 returned error can't find the container with id 870c223e98ac7230aa82ae0ccc2fb53dcec6af9e52070cf838368c721d0a9049 Jul 23 13:13:21.960404 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:21.960371 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz" event={"ID":"e9f3163a-e8ec-4cd6-9a15-17c6310f3d76","Type":"ContainerStarted","Data":"298a92652914829b0cd11f6be9b201984d77cabe98c38345c5cd7b054e0e4225"} Jul 23 13:13:21.960563 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:21.960423 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz" event={"ID":"e9f3163a-e8ec-4cd6-9a15-17c6310f3d76","Type":"ContainerStarted","Data":"870c223e98ac7230aa82ae0ccc2fb53dcec6af9e52070cf838368c721d0a9049"} Jul 23 13:13:22.969340 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:22.969250 2531 generic.go:356] "Generic (PLEG): container finished" podID="e9f3163a-e8ec-4cd6-9a15-17c6310f3d76" containerID="298a92652914829b0cd11f6be9b201984d77cabe98c38345c5cd7b054e0e4225" exitCode=0 Jul 23 13:13:22.969340 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:22.969331 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz" event={"ID":"e9f3163a-e8ec-4cd6-9a15-17c6310f3d76","Type":"ContainerDied","Data":"298a92652914829b0cd11f6be9b201984d77cabe98c38345c5cd7b054e0e4225"} Jul 23 13:13:23.975647 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:23.975560 2531 generic.go:356] "Generic (PLEG): container finished" podID="e9f3163a-e8ec-4cd6-9a15-17c6310f3d76" containerID="e91903040947cf9ec18134934bf9e68948921e6e918f7dbbb40eb9b141a83431" exitCode=0 Jul 23 13:13:23.975647 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:23.975619 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz" event={"ID":"e9f3163a-e8ec-4cd6-9a15-17c6310f3d76","Type":"ContainerDied","Data":"e91903040947cf9ec18134934bf9e68948921e6e918f7dbbb40eb9b141a83431"} Jul 23 13:13:24.982431 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:24.982397 2531 generic.go:356] "Generic (PLEG): container finished" podID="e9f3163a-e8ec-4cd6-9a15-17c6310f3d76" containerID="1d4c7ef349d1dbd6ab5c92dfd31b76aa031eec8f682121b04aabf619c1111273" exitCode=0 Jul 23 13:13:24.982431 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:24.982434 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz" event={"ID":"e9f3163a-e8ec-4cd6-9a15-17c6310f3d76","Type":"ContainerDied","Data":"1d4c7ef349d1dbd6ab5c92dfd31b76aa031eec8f682121b04aabf619c1111273"} Jul 23 13:13:26.112276 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:26.112254 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz" Jul 23 13:13:26.207518 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:26.207489 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9f3163a-e8ec-4cd6-9a15-17c6310f3d76-util\") pod \"e9f3163a-e8ec-4cd6-9a15-17c6310f3d76\" (UID: \"e9f3163a-e8ec-4cd6-9a15-17c6310f3d76\") " Jul 23 13:13:26.207687 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:26.207541 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bghdm\" (UniqueName: \"kubernetes.io/projected/e9f3163a-e8ec-4cd6-9a15-17c6310f3d76-kube-api-access-bghdm\") pod \"e9f3163a-e8ec-4cd6-9a15-17c6310f3d76\" (UID: \"e9f3163a-e8ec-4cd6-9a15-17c6310f3d76\") " Jul 23 13:13:26.207687 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:26.207574 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9f3163a-e8ec-4cd6-9a15-17c6310f3d76-bundle\") pod \"e9f3163a-e8ec-4cd6-9a15-17c6310f3d76\" (UID: \"e9f3163a-e8ec-4cd6-9a15-17c6310f3d76\") " Jul 23 13:13:26.208312 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:26.208285 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9f3163a-e8ec-4cd6-9a15-17c6310f3d76-bundle" (OuterVolumeSpecName: "bundle") pod "e9f3163a-e8ec-4cd6-9a15-17c6310f3d76" (UID: "e9f3163a-e8ec-4cd6-9a15-17c6310f3d76"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:13:26.209765 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:26.209740 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9f3163a-e8ec-4cd6-9a15-17c6310f3d76-kube-api-access-bghdm" (OuterVolumeSpecName: "kube-api-access-bghdm") pod "e9f3163a-e8ec-4cd6-9a15-17c6310f3d76" (UID: "e9f3163a-e8ec-4cd6-9a15-17c6310f3d76"). InnerVolumeSpecName "kube-api-access-bghdm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:13:26.216133 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:26.216109 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9f3163a-e8ec-4cd6-9a15-17c6310f3d76-util" (OuterVolumeSpecName: "util") pod "e9f3163a-e8ec-4cd6-9a15-17c6310f3d76" (UID: "e9f3163a-e8ec-4cd6-9a15-17c6310f3d76"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:13:26.308765 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:26.308737 2531 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9f3163a-e8ec-4cd6-9a15-17c6310f3d76-util\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:13:26.308765 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:26.308766 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bghdm\" (UniqueName: \"kubernetes.io/projected/e9f3163a-e8ec-4cd6-9a15-17c6310f3d76-kube-api-access-bghdm\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:13:26.308946 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:26.308777 2531 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9f3163a-e8ec-4cd6-9a15-17c6310f3d76-bundle\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:13:26.992726 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:26.992690 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz" event={"ID":"e9f3163a-e8ec-4cd6-9a15-17c6310f3d76","Type":"ContainerDied","Data":"870c223e98ac7230aa82ae0ccc2fb53dcec6af9e52070cf838368c721d0a9049"} Jul 23 13:13:26.992912 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:26.992751 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="870c223e98ac7230aa82ae0ccc2fb53dcec6af9e52070cf838368c721d0a9049" Jul 23 13:13:26.992912 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:26.992709 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflstgz" Jul 23 13:13:34.403393 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.403282 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k"] Jul 23 13:13:34.403920 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.403738 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9f3163a-e8ec-4cd6-9a15-17c6310f3d76" containerName="pull" Jul 23 13:13:34.403920 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.403757 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9f3163a-e8ec-4cd6-9a15-17c6310f3d76" containerName="pull" Jul 23 13:13:34.403920 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.403786 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9f3163a-e8ec-4cd6-9a15-17c6310f3d76" containerName="extract" Jul 23 13:13:34.403920 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.403794 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9f3163a-e8ec-4cd6-9a15-17c6310f3d76" containerName="extract" Jul 23 13:13:34.403920 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.403805 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9f3163a-e8ec-4cd6-9a15-17c6310f3d76" containerName="util" Jul 23 13:13:34.403920 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.403814 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9f3163a-e8ec-4cd6-9a15-17c6310f3d76" containerName="util" Jul 23 13:13:34.403920 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.403892 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9f3163a-e8ec-4cd6-9a15-17c6310f3d76" containerName="extract" Jul 23 13:13:34.407031 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.407013 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k" Jul 23 13:13:34.413325 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.413305 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 23 13:13:34.413899 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.413329 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 23 13:13:34.413899 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.413406 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 23 13:13:34.414070 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.413532 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-md9mv\"" Jul 23 13:13:34.414070 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.413622 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 23 13:13:34.437604 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.437573 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k"] Jul 23 13:13:34.585836 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.585802 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0271e73f-acbc-4888-b98b-6d3883738a3f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-779cfbfdf-tb82k\" (UID: \"0271e73f-acbc-4888-b98b-6d3883738a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k" Jul 23 13:13:34.586016 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.585852 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0271e73f-acbc-4888-b98b-6d3883738a3f-manifests\") pod \"opendatahub-operator-controller-manager-779cfbfdf-tb82k\" (UID: \"0271e73f-acbc-4888-b98b-6d3883738a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k" Jul 23 13:13:34.586016 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.585890 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25j9f\" (UniqueName: \"kubernetes.io/projected/0271e73f-acbc-4888-b98b-6d3883738a3f-kube-api-access-25j9f\") pod \"opendatahub-operator-controller-manager-779cfbfdf-tb82k\" (UID: \"0271e73f-acbc-4888-b98b-6d3883738a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k" Jul 23 13:13:34.586016 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.585947 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0271e73f-acbc-4888-b98b-6d3883738a3f-tmp\") pod \"opendatahub-operator-controller-manager-779cfbfdf-tb82k\" (UID: \"0271e73f-acbc-4888-b98b-6d3883738a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k" Jul 23 13:13:34.586016 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.585991 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0271e73f-acbc-4888-b98b-6d3883738a3f-webhook-cert\") pod \"opendatahub-operator-controller-manager-779cfbfdf-tb82k\" (UID: \"0271e73f-acbc-4888-b98b-6d3883738a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k" Jul 23 13:13:34.687187 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.687100 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0271e73f-acbc-4888-b98b-6d3883738a3f-manifests\") pod \"opendatahub-operator-controller-manager-779cfbfdf-tb82k\" (UID: \"0271e73f-acbc-4888-b98b-6d3883738a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k" Jul 23 13:13:34.687187 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.687147 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25j9f\" (UniqueName: \"kubernetes.io/projected/0271e73f-acbc-4888-b98b-6d3883738a3f-kube-api-access-25j9f\") pod \"opendatahub-operator-controller-manager-779cfbfdf-tb82k\" (UID: \"0271e73f-acbc-4888-b98b-6d3883738a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k" Jul 23 13:13:34.687187 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.687173 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0271e73f-acbc-4888-b98b-6d3883738a3f-tmp\") pod \"opendatahub-operator-controller-manager-779cfbfdf-tb82k\" (UID: \"0271e73f-acbc-4888-b98b-6d3883738a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k" Jul 23 13:13:34.687485 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.687208 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0271e73f-acbc-4888-b98b-6d3883738a3f-webhook-cert\") pod \"opendatahub-operator-controller-manager-779cfbfdf-tb82k\" (UID: \"0271e73f-acbc-4888-b98b-6d3883738a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k" Jul 23 13:13:34.687485 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.687237 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0271e73f-acbc-4888-b98b-6d3883738a3f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-779cfbfdf-tb82k\" (UID: \"0271e73f-acbc-4888-b98b-6d3883738a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k" Jul 23 13:13:34.687878 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.687850 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0271e73f-acbc-4888-b98b-6d3883738a3f-manifests\") pod \"opendatahub-operator-controller-manager-779cfbfdf-tb82k\" (UID: \"0271e73f-acbc-4888-b98b-6d3883738a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k" Jul 23 13:13:34.689604 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.689574 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0271e73f-acbc-4888-b98b-6d3883738a3f-tmp\") pod \"opendatahub-operator-controller-manager-779cfbfdf-tb82k\" (UID: \"0271e73f-acbc-4888-b98b-6d3883738a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k" Jul 23 13:13:34.690024 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.690005 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0271e73f-acbc-4888-b98b-6d3883738a3f-webhook-cert\") pod \"opendatahub-operator-controller-manager-779cfbfdf-tb82k\" (UID: \"0271e73f-acbc-4888-b98b-6d3883738a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k" Jul 23 13:13:34.690107 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.690018 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0271e73f-acbc-4888-b98b-6d3883738a3f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-779cfbfdf-tb82k\" (UID: \"0271e73f-acbc-4888-b98b-6d3883738a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k" Jul 23 13:13:34.701180 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.701157 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25j9f\" (UniqueName: \"kubernetes.io/projected/0271e73f-acbc-4888-b98b-6d3883738a3f-kube-api-access-25j9f\") pod \"opendatahub-operator-controller-manager-779cfbfdf-tb82k\" (UID: \"0271e73f-acbc-4888-b98b-6d3883738a3f\") " pod="opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k" Jul 23 13:13:34.717248 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.717229 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k" Jul 23 13:13:34.867726 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:34.867700 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k"] Jul 23 13:13:34.869322 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:13:34.869293 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0271e73f_acbc_4888_b98b_6d3883738a3f.slice/crio-845b8cff4c57e1812249076a1c5e714e5ae418da591509cebdb44630ddb8b20e WatchSource:0}: Error finding container 845b8cff4c57e1812249076a1c5e714e5ae418da591509cebdb44630ddb8b20e: Status 404 returned error can't find the container with id 845b8cff4c57e1812249076a1c5e714e5ae418da591509cebdb44630ddb8b20e Jul 23 13:13:35.027672 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:35.027584 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k" event={"ID":"0271e73f-acbc-4888-b98b-6d3883738a3f","Type":"ContainerStarted","Data":"845b8cff4c57e1812249076a1c5e714e5ae418da591509cebdb44630ddb8b20e"} Jul 23 13:13:41.059685 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:41.059645 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k" event={"ID":"0271e73f-acbc-4888-b98b-6d3883738a3f","Type":"ContainerStarted","Data":"ab44564b1a6c59e09d1cb62797c8b0014cc20f94216e8e231d8352d7a486a9d7"} Jul 23 13:13:42.065717 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:42.065683 2531 generic.go:356] "Generic (PLEG): container finished" podID="0271e73f-acbc-4888-b98b-6d3883738a3f" containerID="ab44564b1a6c59e09d1cb62797c8b0014cc20f94216e8e231d8352d7a486a9d7" exitCode=0 Jul 23 13:13:42.066088 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:42.065726 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k" event={"ID":"0271e73f-acbc-4888-b98b-6d3883738a3f","Type":"ContainerDied","Data":"ab44564b1a6c59e09d1cb62797c8b0014cc20f94216e8e231d8352d7a486a9d7"} Jul 23 13:13:44.077373 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:44.077317 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k" event={"ID":"0271e73f-acbc-4888-b98b-6d3883738a3f","Type":"ContainerStarted","Data":"4f2b80923d60106ca3cac2fa5a0c7821754b39cee30710afb86b42927d9b1c10"} Jul 23 13:13:44.077763 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:44.077441 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k" Jul 23 13:13:44.101321 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:44.101274 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k" podStartSLOduration=2.00712732 podStartE2EDuration="10.101257664s" totalImagesPullingTime="6.919144105s" totalInitContainerRuntime="1.174986239s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:13:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:13:44.099210854 +0000 UTC m=+361.338486832" watchObservedRunningTime="2026-07-23 13:13:44.101257664 +0000 UTC m=+361.340533640" Jul 23 13:13:55.085457 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:55.085428 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-779cfbfdf-tb82k" Jul 23 13:13:57.532893 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.532855 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-69f7f7699f-rrm9m"] Jul 23 13:13:57.538338 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.538321 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-69f7f7699f-rrm9m" Jul 23 13:13:57.542036 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.542013 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 23 13:13:57.542125 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.542020 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-4vrkk\"" Jul 23 13:13:57.542212 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.542199 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 23 13:13:57.542768 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.542754 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 23 13:13:57.545240 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.545221 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 23 13:13:57.555609 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.555583 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-69f7f7699f-rrm9m"] Jul 23 13:13:57.582107 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.582073 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f31a5df6-3f63-4aaa-b72e-23628e309ca8-tmp\") pod \"kube-auth-proxy-69f7f7699f-rrm9m\" (UID: \"f31a5df6-3f63-4aaa-b72e-23628e309ca8\") " pod="openshift-ingress/kube-auth-proxy-69f7f7699f-rrm9m" Jul 23 13:13:57.582227 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.582173 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f31a5df6-3f63-4aaa-b72e-23628e309ca8-tls-certs\") pod \"kube-auth-proxy-69f7f7699f-rrm9m\" (UID: \"f31a5df6-3f63-4aaa-b72e-23628e309ca8\") " pod="openshift-ingress/kube-auth-proxy-69f7f7699f-rrm9m" Jul 23 13:13:57.582227 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.582208 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpk7l\" (UniqueName: \"kubernetes.io/projected/f31a5df6-3f63-4aaa-b72e-23628e309ca8-kube-api-access-kpk7l\") pod \"kube-auth-proxy-69f7f7699f-rrm9m\" (UID: \"f31a5df6-3f63-4aaa-b72e-23628e309ca8\") " pod="openshift-ingress/kube-auth-proxy-69f7f7699f-rrm9m" Jul 23 13:13:57.635257 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.635223 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fmspk"] Jul 23 13:13:57.637525 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.637511 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-fmspk" Jul 23 13:13:57.641204 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.641188 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 23 13:13:57.641293 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.641195 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-lkpxb\"" Jul 23 13:13:57.655186 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.655159 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fmspk"] Jul 23 13:13:57.682754 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.682727 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f31a5df6-3f63-4aaa-b72e-23628e309ca8-tmp\") pod \"kube-auth-proxy-69f7f7699f-rrm9m\" (UID: \"f31a5df6-3f63-4aaa-b72e-23628e309ca8\") " pod="openshift-ingress/kube-auth-proxy-69f7f7699f-rrm9m" Jul 23 13:13:57.682888 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.682777 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdzzt\" (UniqueName: \"kubernetes.io/projected/c2c9af72-c32a-4bba-b76d-d31c9614a6da-kube-api-access-jdzzt\") pod \"odh-model-controller-7998c67dc4-fmspk\" (UID: \"c2c9af72-c32a-4bba-b76d-d31c9614a6da\") " pod="opendatahub/odh-model-controller-7998c67dc4-fmspk" Jul 23 13:13:57.682888 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.682811 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f31a5df6-3f63-4aaa-b72e-23628e309ca8-tls-certs\") pod \"kube-auth-proxy-69f7f7699f-rrm9m\" (UID: \"f31a5df6-3f63-4aaa-b72e-23628e309ca8\") " pod="openshift-ingress/kube-auth-proxy-69f7f7699f-rrm9m" Jul 23 13:13:57.682888 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.682837 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpk7l\" (UniqueName: \"kubernetes.io/projected/f31a5df6-3f63-4aaa-b72e-23628e309ca8-kube-api-access-kpk7l\") pod \"kube-auth-proxy-69f7f7699f-rrm9m\" (UID: \"f31a5df6-3f63-4aaa-b72e-23628e309ca8\") " pod="openshift-ingress/kube-auth-proxy-69f7f7699f-rrm9m" Jul 23 13:13:57.682888 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.682860 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2c9af72-c32a-4bba-b76d-d31c9614a6da-cert\") pod \"odh-model-controller-7998c67dc4-fmspk\" (UID: \"c2c9af72-c32a-4bba-b76d-d31c9614a6da\") " pod="opendatahub/odh-model-controller-7998c67dc4-fmspk" Jul 23 13:13:57.685159 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.685138 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f31a5df6-3f63-4aaa-b72e-23628e309ca8-tmp\") pod \"kube-auth-proxy-69f7f7699f-rrm9m\" (UID: \"f31a5df6-3f63-4aaa-b72e-23628e309ca8\") " pod="openshift-ingress/kube-auth-proxy-69f7f7699f-rrm9m" Jul 23 13:13:57.685259 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.685249 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f31a5df6-3f63-4aaa-b72e-23628e309ca8-tls-certs\") pod \"kube-auth-proxy-69f7f7699f-rrm9m\" (UID: \"f31a5df6-3f63-4aaa-b72e-23628e309ca8\") " pod="openshift-ingress/kube-auth-proxy-69f7f7699f-rrm9m" Jul 23 13:13:57.698443 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.698422 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpk7l\" (UniqueName: \"kubernetes.io/projected/f31a5df6-3f63-4aaa-b72e-23628e309ca8-kube-api-access-kpk7l\") pod \"kube-auth-proxy-69f7f7699f-rrm9m\" (UID: \"f31a5df6-3f63-4aaa-b72e-23628e309ca8\") " pod="openshift-ingress/kube-auth-proxy-69f7f7699f-rrm9m" Jul 23 13:13:57.783911 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.783837 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdzzt\" (UniqueName: \"kubernetes.io/projected/c2c9af72-c32a-4bba-b76d-d31c9614a6da-kube-api-access-jdzzt\") pod \"odh-model-controller-7998c67dc4-fmspk\" (UID: \"c2c9af72-c32a-4bba-b76d-d31c9614a6da\") " pod="opendatahub/odh-model-controller-7998c67dc4-fmspk" Jul 23 13:13:57.783911 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.783901 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2c9af72-c32a-4bba-b76d-d31c9614a6da-cert\") pod \"odh-model-controller-7998c67dc4-fmspk\" (UID: \"c2c9af72-c32a-4bba-b76d-d31c9614a6da\") " pod="opendatahub/odh-model-controller-7998c67dc4-fmspk" Jul 23 13:13:57.784088 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:13:57.783999 2531 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 23 13:13:57.784088 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:13:57.784062 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2c9af72-c32a-4bba-b76d-d31c9614a6da-cert podName:c2c9af72-c32a-4bba-b76d-d31c9614a6da nodeName:}" failed. No retries permitted until 2026-07-23 13:13:58.284044841 +0000 UTC m=+375.523320795 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c2c9af72-c32a-4bba-b76d-d31c9614a6da-cert") pod "odh-model-controller-7998c67dc4-fmspk" (UID: "c2c9af72-c32a-4bba-b76d-d31c9614a6da") : secret "odh-model-controller-webhook-cert" not found Jul 23 13:13:57.816478 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.816450 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdzzt\" (UniqueName: \"kubernetes.io/projected/c2c9af72-c32a-4bba-b76d-d31c9614a6da-kube-api-access-jdzzt\") pod \"odh-model-controller-7998c67dc4-fmspk\" (UID: \"c2c9af72-c32a-4bba-b76d-d31c9614a6da\") " pod="opendatahub/odh-model-controller-7998c67dc4-fmspk" Jul 23 13:13:57.847224 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.847196 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-69f7f7699f-rrm9m" Jul 23 13:13:57.977531 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:57.977505 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-69f7f7699f-rrm9m"] Jul 23 13:13:57.980147 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:13:57.980118 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf31a5df6_3f63_4aaa_b72e_23628e309ca8.slice/crio-17228a10a6f5ec859e2ca894f733d1037920605dc904d8f6e8838ebe7d870dfb WatchSource:0}: Error finding container 17228a10a6f5ec859e2ca894f733d1037920605dc904d8f6e8838ebe7d870dfb: Status 404 returned error can't find the container with id 17228a10a6f5ec859e2ca894f733d1037920605dc904d8f6e8838ebe7d870dfb Jul 23 13:13:58.141682 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:58.141649 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-69f7f7699f-rrm9m" event={"ID":"f31a5df6-3f63-4aaa-b72e-23628e309ca8","Type":"ContainerStarted","Data":"17228a10a6f5ec859e2ca894f733d1037920605dc904d8f6e8838ebe7d870dfb"} Jul 23 13:13:58.287696 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:58.287665 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2c9af72-c32a-4bba-b76d-d31c9614a6da-cert\") pod \"odh-model-controller-7998c67dc4-fmspk\" (UID: \"c2c9af72-c32a-4bba-b76d-d31c9614a6da\") " pod="opendatahub/odh-model-controller-7998c67dc4-fmspk" Jul 23 13:13:58.290135 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:58.290112 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2c9af72-c32a-4bba-b76d-d31c9614a6da-cert\") pod \"odh-model-controller-7998c67dc4-fmspk\" (UID: \"c2c9af72-c32a-4bba-b76d-d31c9614a6da\") " pod="opendatahub/odh-model-controller-7998c67dc4-fmspk" Jul 23 13:13:58.546875 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:58.546788 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-fmspk" Jul 23 13:13:58.688200 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:58.688168 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-8gswn"] Jul 23 13:13:58.691016 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:58.690997 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-8gswn" Jul 23 13:13:58.695094 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:58.695072 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 23 13:13:58.702678 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:58.702498 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-zg9m2\"" Jul 23 13:13:58.721596 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:58.721573 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fmspk"] Jul 23 13:13:58.723953 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:13:58.723931 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2c9af72_c32a_4bba_b76d_d31c9614a6da.slice/crio-37e474f1604b6438a772c6c645cfe82a319b4341c3a9e7780d7f638bdd8ffc81 WatchSource:0}: Error finding container 37e474f1604b6438a772c6c645cfe82a319b4341c3a9e7780d7f638bdd8ffc81: Status 404 returned error can't find the container with id 37e474f1604b6438a772c6c645cfe82a319b4341c3a9e7780d7f638bdd8ffc81 Jul 23 13:13:58.733741 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:58.733715 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-8gswn"] Jul 23 13:13:58.791964 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:58.791933 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2h79\" (UniqueName: \"kubernetes.io/projected/90d7ef40-9d2d-4dcf-902f-53f243bec2d1-kube-api-access-h2h79\") pod \"model-serving-api-54d89bd5fc-8gswn\" (UID: \"90d7ef40-9d2d-4dcf-902f-53f243bec2d1\") " pod="opendatahub/model-serving-api-54d89bd5fc-8gswn" Jul 23 13:13:58.792138 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:58.791975 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90d7ef40-9d2d-4dcf-902f-53f243bec2d1-tls-certs\") pod \"model-serving-api-54d89bd5fc-8gswn\" (UID: \"90d7ef40-9d2d-4dcf-902f-53f243bec2d1\") " pod="opendatahub/model-serving-api-54d89bd5fc-8gswn" Jul 23 13:13:58.893036 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:58.893003 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2h79\" (UniqueName: \"kubernetes.io/projected/90d7ef40-9d2d-4dcf-902f-53f243bec2d1-kube-api-access-h2h79\") pod \"model-serving-api-54d89bd5fc-8gswn\" (UID: \"90d7ef40-9d2d-4dcf-902f-53f243bec2d1\") " pod="opendatahub/model-serving-api-54d89bd5fc-8gswn" Jul 23 13:13:58.893036 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:58.893050 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90d7ef40-9d2d-4dcf-902f-53f243bec2d1-tls-certs\") pod \"model-serving-api-54d89bd5fc-8gswn\" (UID: \"90d7ef40-9d2d-4dcf-902f-53f243bec2d1\") " pod="opendatahub/model-serving-api-54d89bd5fc-8gswn" Jul 23 13:13:58.895871 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:58.895846 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90d7ef40-9d2d-4dcf-902f-53f243bec2d1-tls-certs\") pod \"model-serving-api-54d89bd5fc-8gswn\" (UID: \"90d7ef40-9d2d-4dcf-902f-53f243bec2d1\") " pod="opendatahub/model-serving-api-54d89bd5fc-8gswn" Jul 23 13:13:58.911022 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:58.910993 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2h79\" (UniqueName: \"kubernetes.io/projected/90d7ef40-9d2d-4dcf-902f-53f243bec2d1-kube-api-access-h2h79\") pod \"model-serving-api-54d89bd5fc-8gswn\" (UID: \"90d7ef40-9d2d-4dcf-902f-53f243bec2d1\") " pod="opendatahub/model-serving-api-54d89bd5fc-8gswn" Jul 23 13:13:59.010749 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:59.010711 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-8gswn" Jul 23 13:13:59.149653 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:59.149566 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-fmspk" event={"ID":"c2c9af72-c32a-4bba-b76d-d31c9614a6da","Type":"ContainerStarted","Data":"37e474f1604b6438a772c6c645cfe82a319b4341c3a9e7780d7f638bdd8ffc81"} Jul 23 13:13:59.164721 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:13:59.164668 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-8gswn"] Jul 23 13:13:59.166463 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:13:59.166434 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90d7ef40_9d2d_4dcf_902f_53f243bec2d1.slice/crio-a8a5e544d8e855b0d13c7a10bb0331d651dd2e9283d35a15af9924608bccb96c WatchSource:0}: Error finding container a8a5e544d8e855b0d13c7a10bb0331d651dd2e9283d35a15af9924608bccb96c: Status 404 returned error can't find the container with id a8a5e544d8e855b0d13c7a10bb0331d651dd2e9283d35a15af9924608bccb96c Jul 23 13:14:00.157680 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:00.157434 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-8gswn" event={"ID":"90d7ef40-9d2d-4dcf-902f-53f243bec2d1","Type":"ContainerStarted","Data":"a8a5e544d8e855b0d13c7a10bb0331d651dd2e9283d35a15af9924608bccb96c"} Jul 23 13:14:03.180811 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:03.180771 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-69f7f7699f-rrm9m" event={"ID":"f31a5df6-3f63-4aaa-b72e-23628e309ca8","Type":"ContainerStarted","Data":"fc49501bebe6d34ad8e392e5d192630f916920d4b3fa76763ee2f8028a4a0289"} Jul 23 13:14:03.182253 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:03.182223 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-8gswn" event={"ID":"90d7ef40-9d2d-4dcf-902f-53f243bec2d1","Type":"ContainerStarted","Data":"9ef38804854b31a40a4bf1dfb84b8a18dea695b33646738a1c38f9e397044649"} Jul 23 13:14:03.182407 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:03.182279 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-8gswn" Jul 23 13:14:03.183652 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:03.183625 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-fmspk" event={"ID":"c2c9af72-c32a-4bba-b76d-d31c9614a6da","Type":"ContainerStarted","Data":"737945141b8d20507450894907c20d1bce8034ca930072a635525c63239730ef"} Jul 23 13:14:03.183748 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:03.183676 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-fmspk" Jul 23 13:14:03.198704 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:03.198660 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-69f7f7699f-rrm9m" podStartSLOduration=1.134190348 podStartE2EDuration="6.198646163s" totalImagesPullingTime="5.064455815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:13:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:14:03.197297846 +0000 UTC m=+380.436573824" watchObservedRunningTime="2026-07-23 13:14:03.198646163 +0000 UTC m=+380.437922146" Jul 23 13:14:03.214772 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:03.214726 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-fmspk" podStartSLOduration=1.880155835 podStartE2EDuration="6.214711905s" totalImagesPullingTime="4.33455607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:13:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:14:03.213186014 +0000 UTC m=+380.452462002" watchObservedRunningTime="2026-07-23 13:14:03.214711905 +0000 UTC m=+380.453987883" Jul 23 13:14:03.229987 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:03.229927 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-8gswn" podStartSLOduration=1.375284429 podStartE2EDuration="5.229906948s" totalImagesPullingTime="3.854622519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:13:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:14:03.22897707 +0000 UTC m=+380.468253046" watchObservedRunningTime="2026-07-23 13:14:03.229906948 +0000 UTC m=+380.469182925" Jul 23 13:14:05.277874 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.277843 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx"] Jul 23 13:14:05.280412 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.280392 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.282896 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.282875 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 23 13:14:05.283476 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.283446 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-w4dlz\"" Jul 23 13:14:05.297536 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.297509 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx"] Jul 23 13:14:05.354041 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.354011 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5qtk\" (UniqueName: \"kubernetes.io/projected/56168794-d92a-4663-91dc-babcfc678a53-kube-api-access-c5qtk\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.354199 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.354051 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56168794-d92a-4663-91dc-babcfc678a53-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.354199 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.354078 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56168794-d92a-4663-91dc-babcfc678a53-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.354199 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.354140 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56168794-d92a-4663-91dc-babcfc678a53-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.354304 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.354211 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56168794-d92a-4663-91dc-babcfc678a53-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.354304 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.354253 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56168794-d92a-4663-91dc-babcfc678a53-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.354304 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.354276 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56168794-d92a-4663-91dc-babcfc678a53-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.354436 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.354314 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56168794-d92a-4663-91dc-babcfc678a53-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.354436 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.354340 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56168794-d92a-4663-91dc-babcfc678a53-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.455860 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.455823 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56168794-d92a-4663-91dc-babcfc678a53-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.456066 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.455906 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56168794-d92a-4663-91dc-babcfc678a53-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.456399 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.456342 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56168794-d92a-4663-91dc-babcfc678a53-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.456399 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.456397 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56168794-d92a-4663-91dc-babcfc678a53-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.456596 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.456411 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56168794-d92a-4663-91dc-babcfc678a53-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.456596 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.456480 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56168794-d92a-4663-91dc-babcfc678a53-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.456596 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.456512 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56168794-d92a-4663-91dc-babcfc678a53-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.456596 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.456559 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5qtk\" (UniqueName: \"kubernetes.io/projected/56168794-d92a-4663-91dc-babcfc678a53-kube-api-access-c5qtk\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.456811 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.456620 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56168794-d92a-4663-91dc-babcfc678a53-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.456811 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.456636 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56168794-d92a-4663-91dc-babcfc678a53-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.456811 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.456734 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56168794-d92a-4663-91dc-babcfc678a53-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.456811 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.456774 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56168794-d92a-4663-91dc-babcfc678a53-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.457104 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.457082 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56168794-d92a-4663-91dc-babcfc678a53-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.457272 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.457257 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56168794-d92a-4663-91dc-babcfc678a53-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.458991 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.458970 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56168794-d92a-4663-91dc-babcfc678a53-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.459102 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.459088 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56168794-d92a-4663-91dc-babcfc678a53-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.467433 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.467408 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm"] Jul 23 13:14:05.470870 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.470846 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" Jul 23 13:14:05.471517 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.471496 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56168794-d92a-4663-91dc-babcfc678a53-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.471599 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.471569 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5qtk\" (UniqueName: \"kubernetes.io/projected/56168794-d92a-4663-91dc-babcfc678a53-kube-api-access-c5qtk\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rglnx\" (UID: \"56168794-d92a-4663-91dc-babcfc678a53\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.473147 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.473128 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 23 13:14:05.473250 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.473127 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 23 13:14:05.473250 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.473180 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-sw685\"" Jul 23 13:14:05.473250 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.473148 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 23 13:14:05.485008 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.484984 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm"] Jul 23 13:14:05.557772 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.557748 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/72f97835-f1c9-4b7d-9af5-9ccc616b7cd7-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-x4whm\" (UID: \"72f97835-f1c9-4b7d-9af5-9ccc616b7cd7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" Jul 23 13:14:05.557958 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.557779 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkglg\" (UniqueName: \"kubernetes.io/projected/72f97835-f1c9-4b7d-9af5-9ccc616b7cd7-kube-api-access-mkglg\") pod \"istiod-openshift-gateway-94bb8fbfd-x4whm\" (UID: \"72f97835-f1c9-4b7d-9af5-9ccc616b7cd7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" Jul 23 13:14:05.557958 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.557801 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/72f97835-f1c9-4b7d-9af5-9ccc616b7cd7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-x4whm\" (UID: \"72f97835-f1c9-4b7d-9af5-9ccc616b7cd7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" Jul 23 13:14:05.557958 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.557837 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72f97835-f1c9-4b7d-9af5-9ccc616b7cd7-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-x4whm\" (UID: \"72f97835-f1c9-4b7d-9af5-9ccc616b7cd7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" Jul 23 13:14:05.557958 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.557935 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/72f97835-f1c9-4b7d-9af5-9ccc616b7cd7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-x4whm\" (UID: \"72f97835-f1c9-4b7d-9af5-9ccc616b7cd7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" Jul 23 13:14:05.558116 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.557962 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/72f97835-f1c9-4b7d-9af5-9ccc616b7cd7-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-x4whm\" (UID: \"72f97835-f1c9-4b7d-9af5-9ccc616b7cd7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" Jul 23 13:14:05.558116 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.558033 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72f97835-f1c9-4b7d-9af5-9ccc616b7cd7-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-x4whm\" (UID: \"72f97835-f1c9-4b7d-9af5-9ccc616b7cd7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" Jul 23 13:14:05.594904 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.594869 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:05.660379 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.658791 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/72f97835-f1c9-4b7d-9af5-9ccc616b7cd7-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-x4whm\" (UID: \"72f97835-f1c9-4b7d-9af5-9ccc616b7cd7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" Jul 23 13:14:05.660379 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.658886 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72f97835-f1c9-4b7d-9af5-9ccc616b7cd7-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-x4whm\" (UID: \"72f97835-f1c9-4b7d-9af5-9ccc616b7cd7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" Jul 23 13:14:05.660379 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.658941 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/72f97835-f1c9-4b7d-9af5-9ccc616b7cd7-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-x4whm\" (UID: \"72f97835-f1c9-4b7d-9af5-9ccc616b7cd7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" Jul 23 13:14:05.660379 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.658970 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkglg\" (UniqueName: \"kubernetes.io/projected/72f97835-f1c9-4b7d-9af5-9ccc616b7cd7-kube-api-access-mkglg\") pod \"istiod-openshift-gateway-94bb8fbfd-x4whm\" (UID: \"72f97835-f1c9-4b7d-9af5-9ccc616b7cd7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" Jul 23 13:14:05.660379 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.659007 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/72f97835-f1c9-4b7d-9af5-9ccc616b7cd7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-x4whm\" (UID: \"72f97835-f1c9-4b7d-9af5-9ccc616b7cd7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" Jul 23 13:14:05.660379 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.659059 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72f97835-f1c9-4b7d-9af5-9ccc616b7cd7-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-x4whm\" (UID: \"72f97835-f1c9-4b7d-9af5-9ccc616b7cd7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" Jul 23 13:14:05.660379 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.659100 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/72f97835-f1c9-4b7d-9af5-9ccc616b7cd7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-x4whm\" (UID: \"72f97835-f1c9-4b7d-9af5-9ccc616b7cd7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" Jul 23 13:14:05.663550 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.661101 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/72f97835-f1c9-4b7d-9af5-9ccc616b7cd7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-x4whm\" (UID: \"72f97835-f1c9-4b7d-9af5-9ccc616b7cd7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" Jul 23 13:14:05.663550 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.663093 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/72f97835-f1c9-4b7d-9af5-9ccc616b7cd7-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-x4whm\" (UID: \"72f97835-f1c9-4b7d-9af5-9ccc616b7cd7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" Jul 23 13:14:05.664308 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.663771 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72f97835-f1c9-4b7d-9af5-9ccc616b7cd7-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-x4whm\" (UID: \"72f97835-f1c9-4b7d-9af5-9ccc616b7cd7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" Jul 23 13:14:05.664486 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.664315 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/72f97835-f1c9-4b7d-9af5-9ccc616b7cd7-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-x4whm\" (UID: \"72f97835-f1c9-4b7d-9af5-9ccc616b7cd7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" Jul 23 13:14:05.666323 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.666282 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/72f97835-f1c9-4b7d-9af5-9ccc616b7cd7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-x4whm\" (UID: \"72f97835-f1c9-4b7d-9af5-9ccc616b7cd7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" Jul 23 13:14:05.679918 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.679896 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkglg\" (UniqueName: \"kubernetes.io/projected/72f97835-f1c9-4b7d-9af5-9ccc616b7cd7-kube-api-access-mkglg\") pod \"istiod-openshift-gateway-94bb8fbfd-x4whm\" (UID: \"72f97835-f1c9-4b7d-9af5-9ccc616b7cd7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" Jul 23 13:14:05.680078 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.680005 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72f97835-f1c9-4b7d-9af5-9ccc616b7cd7-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-x4whm\" (UID: \"72f97835-f1c9-4b7d-9af5-9ccc616b7cd7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" Jul 23 13:14:05.771466 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.771437 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx"] Jul 23 13:14:05.772738 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:14:05.772716 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56168794_d92a_4663_91dc_babcfc678a53.slice/crio-c5279c4af5436ac16fc9905fcc621ce39cf6149bc9bf5b6dbbab06a66f101b36 WatchSource:0}: Error finding container c5279c4af5436ac16fc9905fcc621ce39cf6149bc9bf5b6dbbab06a66f101b36: Status 404 returned error can't find the container with id c5279c4af5436ac16fc9905fcc621ce39cf6149bc9bf5b6dbbab06a66f101b36 Jul 23 13:14:05.792603 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.792578 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" Jul 23 13:14:05.923622 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:05.923600 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm"] Jul 23 13:14:05.925925 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:14:05.925898 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72f97835_f1c9_4b7d_9af5_9ccc616b7cd7.slice/crio-4434b1b79818e76840c024f1dca68033ace333cced82411ecd4f9b4954dd1a9e WatchSource:0}: Error finding container 4434b1b79818e76840c024f1dca68033ace333cced82411ecd4f9b4954dd1a9e: Status 404 returned error can't find the container with id 4434b1b79818e76840c024f1dca68033ace333cced82411ecd4f9b4954dd1a9e Jul 23 13:14:06.198550 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:06.198465 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" event={"ID":"56168794-d92a-4663-91dc-babcfc678a53","Type":"ContainerStarted","Data":"c5279c4af5436ac16fc9905fcc621ce39cf6149bc9bf5b6dbbab06a66f101b36"} Jul 23 13:14:06.199770 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:06.199744 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" event={"ID":"72f97835-f1c9-4b7d-9af5-9ccc616b7cd7","Type":"ContainerStarted","Data":"4434b1b79818e76840c024f1dca68033ace333cced82411ecd4f9b4954dd1a9e"} Jul 23 13:14:08.213896 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:08.213854 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:14:08.214133 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:08.213930 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:14:08.951368 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:08.951322 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-x2pgk"] Jul 23 13:14:08.957000 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:08.956979 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2pgk" Jul 23 13:14:08.959509 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:08.959484 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 23 13:14:08.959679 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:08.959661 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-kw5gc\"" Jul 23 13:14:08.976542 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:08.976508 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-x2pgk"] Jul 23 13:14:09.090214 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:09.090183 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbc65\" (UniqueName: \"kubernetes.io/projected/6d067d22-ad1c-414c-8be0-74d3a393b34e-kube-api-access-bbc65\") pod \"kserve-controller-manager-d6f4b65b8-x2pgk\" (UID: \"6d067d22-ad1c-414c-8be0-74d3a393b34e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2pgk" Jul 23 13:14:09.090214 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:09.090227 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d067d22-ad1c-414c-8be0-74d3a393b34e-cert\") pod \"kserve-controller-manager-d6f4b65b8-x2pgk\" (UID: \"6d067d22-ad1c-414c-8be0-74d3a393b34e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2pgk" Jul 23 13:14:09.191670 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:09.191627 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbc65\" (UniqueName: \"kubernetes.io/projected/6d067d22-ad1c-414c-8be0-74d3a393b34e-kube-api-access-bbc65\") pod \"kserve-controller-manager-d6f4b65b8-x2pgk\" (UID: \"6d067d22-ad1c-414c-8be0-74d3a393b34e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2pgk" Jul 23 13:14:09.191828 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:09.191682 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d067d22-ad1c-414c-8be0-74d3a393b34e-cert\") pod \"kserve-controller-manager-d6f4b65b8-x2pgk\" (UID: \"6d067d22-ad1c-414c-8be0-74d3a393b34e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2pgk" Jul 23 13:14:09.194286 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:09.194254 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d067d22-ad1c-414c-8be0-74d3a393b34e-cert\") pod \"kserve-controller-manager-d6f4b65b8-x2pgk\" (UID: \"6d067d22-ad1c-414c-8be0-74d3a393b34e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2pgk" Jul 23 13:14:09.203229 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:09.203169 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbc65\" (UniqueName: \"kubernetes.io/projected/6d067d22-ad1c-414c-8be0-74d3a393b34e-kube-api-access-bbc65\") pod \"kserve-controller-manager-d6f4b65b8-x2pgk\" (UID: \"6d067d22-ad1c-414c-8be0-74d3a393b34e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2pgk" Jul 23 13:14:09.215237 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:09.215213 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" event={"ID":"72f97835-f1c9-4b7d-9af5-9ccc616b7cd7","Type":"ContainerStarted","Data":"f4a75d9cc6090f121d5d0e07167179cc62f0f77935b0f06d35a03c8d629fc0ac"} Jul 23 13:14:09.215576 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:09.215462 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" Jul 23 13:14:09.217054 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:09.217036 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" Jul 23 13:14:09.267216 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:09.267176 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2pgk" Jul 23 13:14:09.268270 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:09.268230 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-x4whm" podStartSLOduration=1.982637116 podStartE2EDuration="4.268217581s" totalImagesPullingTime="2.285580465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:14:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:14:09.265190198 +0000 UTC m=+386.504466174" watchObservedRunningTime="2026-07-23 13:14:09.268217581 +0000 UTC m=+386.507493547" Jul 23 13:14:09.443336 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:09.443258 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-x2pgk"] Jul 23 13:14:09.447271 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:14:09.447239 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d067d22_ad1c_414c_8be0_74d3a393b34e.slice/crio-0a5131cf2d43470358ad385ea2f1dca5f0733080eab51978fab3df42abd68839 WatchSource:0}: Error finding container 0a5131cf2d43470358ad385ea2f1dca5f0733080eab51978fab3df42abd68839: Status 404 returned error can't find the container with id 0a5131cf2d43470358ad385ea2f1dca5f0733080eab51978fab3df42abd68839 Jul 23 13:14:10.222183 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:10.222142 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2pgk" event={"ID":"6d067d22-ad1c-414c-8be0-74d3a393b34e","Type":"ContainerStarted","Data":"0a5131cf2d43470358ad385ea2f1dca5f0733080eab51978fab3df42abd68839"} Jul 23 13:14:11.670727 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:11.670690 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:14:11.671105 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:11.670772 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:14:11.671105 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:11.670821 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:14:12.235301 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:12.235254 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" event={"ID":"56168794-d92a-4663-91dc-babcfc678a53","Type":"ContainerStarted","Data":"f2da5d4b421bfe592357a997469b7ea5d591fc79d05a57c2aa2c02eaf5a42e75"} Jul 23 13:14:12.256993 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:12.256942 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" podStartSLOduration=1.361185359 podStartE2EDuration="7.256926997s" totalImagesPullingTime="5.895741638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:14:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:14:12.25549742 +0000 UTC m=+389.494773399" watchObservedRunningTime="2026-07-23 13:14:12.256926997 +0000 UTC m=+389.496202974" Jul 23 13:14:12.595390 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:12.595332 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:12.600185 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:12.600159 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:13.241343 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:13.241308 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:13.242403 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:13.242380 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rglnx" Jul 23 13:14:14.191328 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:14.191304 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-fmspk" Jul 23 13:14:14.193544 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:14.193525 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-8gswn" Jul 23 13:14:14.247628 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:14.247577 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2pgk" event={"ID":"6d067d22-ad1c-414c-8be0-74d3a393b34e","Type":"ContainerStarted","Data":"84b06b49fe4e4acc5d4bd32a836a593471e465af73cc52ece2e6fc25f14b78a5"} Jul 23 13:14:14.248161 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:14.248138 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2pgk" Jul 23 13:14:14.264567 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:14.264514 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2pgk" podStartSLOduration=2.241353758 podStartE2EDuration="6.264497894s" totalImagesPullingTime="4.023144136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:14:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:14:14.262227756 +0000 UTC m=+391.501503732" watchObservedRunningTime="2026-07-23 13:14:14.264497894 +0000 UTC m=+391.503773870" Jul 23 13:14:46.262031 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:46.261997 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2pgk" Jul 23 13:14:58.405507 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:58.405472 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vwgsl"] Jul 23 13:14:58.418861 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:58.417594 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vwgsl" Jul 23 13:14:58.419642 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:58.419617 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vwgsl"] Jul 23 13:14:58.421269 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:58.421238 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 23 13:14:58.421432 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:58.421346 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 23 13:14:58.422209 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:58.422189 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-mtdfp\"" Jul 23 13:14:58.523751 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:58.523716 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2jjw\" (UniqueName: \"kubernetes.io/projected/33e16f3f-df38-45e6-b18f-1c21826b2102-kube-api-access-f2jjw\") pod \"kuadrant-operator-catalog-vwgsl\" (UID: \"33e16f3f-df38-45e6-b18f-1c21826b2102\") " pod="kuadrant-system/kuadrant-operator-catalog-vwgsl" Jul 23 13:14:58.625135 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:58.625102 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2jjw\" (UniqueName: \"kubernetes.io/projected/33e16f3f-df38-45e6-b18f-1c21826b2102-kube-api-access-f2jjw\") pod \"kuadrant-operator-catalog-vwgsl\" (UID: \"33e16f3f-df38-45e6-b18f-1c21826b2102\") " pod="kuadrant-system/kuadrant-operator-catalog-vwgsl" Jul 23 13:14:58.633148 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:58.633113 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2jjw\" (UniqueName: \"kubernetes.io/projected/33e16f3f-df38-45e6-b18f-1c21826b2102-kube-api-access-f2jjw\") pod \"kuadrant-operator-catalog-vwgsl\" (UID: \"33e16f3f-df38-45e6-b18f-1c21826b2102\") " pod="kuadrant-system/kuadrant-operator-catalog-vwgsl" Jul 23 13:14:58.729427 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:58.729318 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vwgsl" Jul 23 13:14:58.756415 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:58.756382 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vwgsl"] Jul 23 13:14:58.860368 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:58.860304 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vwgsl"] Jul 23 13:14:58.862537 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:14:58.862508 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33e16f3f_df38_45e6_b18f_1c21826b2102.slice/crio-c8972e4ea92925e9bb82e3a4a689f3d0c263db9462471fb96d246a9b0e7e45de WatchSource:0}: Error finding container c8972e4ea92925e9bb82e3a4a689f3d0c263db9462471fb96d246a9b0e7e45de: Status 404 returned error can't find the container with id c8972e4ea92925e9bb82e3a4a689f3d0c263db9462471fb96d246a9b0e7e45de Jul 23 13:14:58.976081 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:58.976052 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b2kkv"] Jul 23 13:14:58.979210 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:58.979193 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-b2kkv" Jul 23 13:14:59.003593 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:59.003565 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b2kkv"] Jul 23 13:14:59.028313 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:59.028284 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcwbg\" (UniqueName: \"kubernetes.io/projected/9bdec0ad-3a77-4067-95ca-d13d7a3724ae-kube-api-access-wcwbg\") pod \"kuadrant-operator-catalog-b2kkv\" (UID: \"9bdec0ad-3a77-4067-95ca-d13d7a3724ae\") " pod="kuadrant-system/kuadrant-operator-catalog-b2kkv" Jul 23 13:14:59.129233 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:59.129202 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcwbg\" (UniqueName: \"kubernetes.io/projected/9bdec0ad-3a77-4067-95ca-d13d7a3724ae-kube-api-access-wcwbg\") pod \"kuadrant-operator-catalog-b2kkv\" (UID: \"9bdec0ad-3a77-4067-95ca-d13d7a3724ae\") " pod="kuadrant-system/kuadrant-operator-catalog-b2kkv" Jul 23 13:14:59.138666 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:59.138639 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcwbg\" (UniqueName: \"kubernetes.io/projected/9bdec0ad-3a77-4067-95ca-d13d7a3724ae-kube-api-access-wcwbg\") pod \"kuadrant-operator-catalog-b2kkv\" (UID: \"9bdec0ad-3a77-4067-95ca-d13d7a3724ae\") " pod="kuadrant-system/kuadrant-operator-catalog-b2kkv" Jul 23 13:14:59.288687 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:59.288600 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-b2kkv" Jul 23 13:14:59.420343 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:59.420305 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b2kkv"] Jul 23 13:14:59.424602 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:14:59.424564 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bdec0ad_3a77_4067_95ca_d13d7a3724ae.slice/crio-c50cfb669e6222353ba63ab3528a88807d10f8266574d61ca58b8c5aedf99d3b WatchSource:0}: Error finding container c50cfb669e6222353ba63ab3528a88807d10f8266574d61ca58b8c5aedf99d3b: Status 404 returned error can't find the container with id c50cfb669e6222353ba63ab3528a88807d10f8266574d61ca58b8c5aedf99d3b Jul 23 13:14:59.454731 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:59.454698 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-b2kkv" event={"ID":"9bdec0ad-3a77-4067-95ca-d13d7a3724ae","Type":"ContainerStarted","Data":"c50cfb669e6222353ba63ab3528a88807d10f8266574d61ca58b8c5aedf99d3b"} Jul 23 13:14:59.455797 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:14:59.455776 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vwgsl" event={"ID":"33e16f3f-df38-45e6-b18f-1c21826b2102","Type":"ContainerStarted","Data":"c8972e4ea92925e9bb82e3a4a689f3d0c263db9462471fb96d246a9b0e7e45de"} Jul 23 13:15:02.473583 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:02.473540 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vwgsl" event={"ID":"33e16f3f-df38-45e6-b18f-1c21826b2102","Type":"ContainerStarted","Data":"5c1c9fc729d9bc64b10828706b25ef6f164c7df86037d05351ae4770ecef0405"} Jul 23 13:15:02.474036 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:02.473590 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-vwgsl" podUID="33e16f3f-df38-45e6-b18f-1c21826b2102" containerName="registry-server" containerID="cri-o://5c1c9fc729d9bc64b10828706b25ef6f164c7df86037d05351ae4770ecef0405" gracePeriod=2 Jul 23 13:15:02.475249 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:02.475220 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-b2kkv" event={"ID":"9bdec0ad-3a77-4067-95ca-d13d7a3724ae","Type":"ContainerStarted","Data":"84d8f0911172ba56f928808fc9052edfb74acbb50c390585be7c2f5d77f84c86"} Jul 23 13:15:02.490614 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:02.490574 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-vwgsl" podStartSLOduration=1.689303075 podStartE2EDuration="4.490556959s" totalImagesPullingTime="2.801253884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:14:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:15:02.488069435 +0000 UTC m=+439.727345419" watchObservedRunningTime="2026-07-23 13:15:02.490556959 +0000 UTC m=+439.729832965" Jul 23 13:15:02.504698 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:02.504653 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-b2kkv" podStartSLOduration=2.268637392 podStartE2EDuration="4.504635856s" totalImagesPullingTime="2.235998464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:14:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:15:02.50330863 +0000 UTC m=+439.742584619" watchObservedRunningTime="2026-07-23 13:15:02.504635856 +0000 UTC m=+439.743911866" Jul 23 13:15:02.719165 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:02.719137 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vwgsl" Jul 23 13:15:02.763994 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:02.763916 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2jjw\" (UniqueName: \"kubernetes.io/projected/33e16f3f-df38-45e6-b18f-1c21826b2102-kube-api-access-f2jjw\") pod \"33e16f3f-df38-45e6-b18f-1c21826b2102\" (UID: \"33e16f3f-df38-45e6-b18f-1c21826b2102\") " Jul 23 13:15:02.766140 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:02.766109 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33e16f3f-df38-45e6-b18f-1c21826b2102-kube-api-access-f2jjw" (OuterVolumeSpecName: "kube-api-access-f2jjw") pod "33e16f3f-df38-45e6-b18f-1c21826b2102" (UID: "33e16f3f-df38-45e6-b18f-1c21826b2102"). InnerVolumeSpecName "kube-api-access-f2jjw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:15:02.865519 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:02.865477 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2jjw\" (UniqueName: \"kubernetes.io/projected/33e16f3f-df38-45e6-b18f-1c21826b2102-kube-api-access-f2jjw\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:15:03.481257 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:03.481178 2531 generic.go:356] "Generic (PLEG): container finished" podID="33e16f3f-df38-45e6-b18f-1c21826b2102" containerID="5c1c9fc729d9bc64b10828706b25ef6f164c7df86037d05351ae4770ecef0405" exitCode=0 Jul 23 13:15:03.481257 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:03.481223 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vwgsl" Jul 23 13:15:03.481781 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:03.481266 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vwgsl" event={"ID":"33e16f3f-df38-45e6-b18f-1c21826b2102","Type":"ContainerDied","Data":"5c1c9fc729d9bc64b10828706b25ef6f164c7df86037d05351ae4770ecef0405"} Jul 23 13:15:03.481781 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:03.481329 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vwgsl" event={"ID":"33e16f3f-df38-45e6-b18f-1c21826b2102","Type":"ContainerDied","Data":"c8972e4ea92925e9bb82e3a4a689f3d0c263db9462471fb96d246a9b0e7e45de"} Jul 23 13:15:03.481781 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:03.481378 2531 scope.go:117] "RemoveContainer" containerID="5c1c9fc729d9bc64b10828706b25ef6f164c7df86037d05351ae4770ecef0405" Jul 23 13:15:03.490617 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:03.490599 2531 scope.go:117] "RemoveContainer" containerID="5c1c9fc729d9bc64b10828706b25ef6f164c7df86037d05351ae4770ecef0405" Jul 23 13:15:03.490913 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:15:03.490893 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c1c9fc729d9bc64b10828706b25ef6f164c7df86037d05351ae4770ecef0405\": container with ID starting with 5c1c9fc729d9bc64b10828706b25ef6f164c7df86037d05351ae4770ecef0405 not found: ID does not exist" containerID="5c1c9fc729d9bc64b10828706b25ef6f164c7df86037d05351ae4770ecef0405" Jul 23 13:15:03.490956 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:03.490924 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c1c9fc729d9bc64b10828706b25ef6f164c7df86037d05351ae4770ecef0405"} err="failed to get container status \"5c1c9fc729d9bc64b10828706b25ef6f164c7df86037d05351ae4770ecef0405\": rpc error: code = NotFound desc = could not find container \"5c1c9fc729d9bc64b10828706b25ef6f164c7df86037d05351ae4770ecef0405\": container with ID starting with 5c1c9fc729d9bc64b10828706b25ef6f164c7df86037d05351ae4770ecef0405 not found: ID does not exist" Jul 23 13:15:03.498607 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:03.498581 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vwgsl"] Jul 23 13:15:03.502315 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:03.502289 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vwgsl"] Jul 23 13:15:05.331873 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:05.331826 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33e16f3f-df38-45e6-b18f-1c21826b2102" path="/var/lib/kubelet/pods/33e16f3f-df38-45e6-b18f-1c21826b2102/volumes" Jul 23 13:15:09.289041 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:09.289007 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-b2kkv" Jul 23 13:15:09.289512 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:09.289231 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-b2kkv" Jul 23 13:15:09.311729 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:09.311704 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-b2kkv" Jul 23 13:15:09.532307 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:09.532283 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-b2kkv" Jul 23 13:15:13.990241 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:13.990204 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq"] Jul 23 13:15:13.990668 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:13.990580 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33e16f3f-df38-45e6-b18f-1c21826b2102" containerName="registry-server" Jul 23 13:15:13.990668 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:13.990591 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="33e16f3f-df38-45e6-b18f-1c21826b2102" containerName="registry-server" Jul 23 13:15:13.990668 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:13.990648 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="33e16f3f-df38-45e6-b18f-1c21826b2102" containerName="registry-server" Jul 23 13:15:13.993986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:13.993968 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq" Jul 23 13:15:13.996224 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:13.996206 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5jhxm\"" Jul 23 13:15:14.002468 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.002446 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq"] Jul 23 13:15:14.071951 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.071916 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfmft\" (UniqueName: \"kubernetes.io/projected/b2bae3f4-18e4-4201-a547-e6850c782cea-kube-api-access-pfmft\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq\" (UID: \"b2bae3f4-18e4-4201-a547-e6850c782cea\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq" Jul 23 13:15:14.072123 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.071964 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b2bae3f4-18e4-4201-a547-e6850c782cea-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq\" (UID: \"b2bae3f4-18e4-4201-a547-e6850c782cea\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq" Jul 23 13:15:14.072123 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.072056 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b2bae3f4-18e4-4201-a547-e6850c782cea-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq\" (UID: \"b2bae3f4-18e4-4201-a547-e6850c782cea\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq" Jul 23 13:15:14.173716 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.173686 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b2bae3f4-18e4-4201-a547-e6850c782cea-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq\" (UID: \"b2bae3f4-18e4-4201-a547-e6850c782cea\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq" Jul 23 13:15:14.173860 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.173777 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfmft\" (UniqueName: \"kubernetes.io/projected/b2bae3f4-18e4-4201-a547-e6850c782cea-kube-api-access-pfmft\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq\" (UID: \"b2bae3f4-18e4-4201-a547-e6850c782cea\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq" Jul 23 13:15:14.173860 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.173820 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b2bae3f4-18e4-4201-a547-e6850c782cea-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq\" (UID: \"b2bae3f4-18e4-4201-a547-e6850c782cea\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq" Jul 23 13:15:14.174078 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.174054 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b2bae3f4-18e4-4201-a547-e6850c782cea-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq\" (UID: \"b2bae3f4-18e4-4201-a547-e6850c782cea\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq" Jul 23 13:15:14.174189 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.174163 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b2bae3f4-18e4-4201-a547-e6850c782cea-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq\" (UID: \"b2bae3f4-18e4-4201-a547-e6850c782cea\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq" Jul 23 13:15:14.182748 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.182721 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfmft\" (UniqueName: \"kubernetes.io/projected/b2bae3f4-18e4-4201-a547-e6850c782cea-kube-api-access-pfmft\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq\" (UID: \"b2bae3f4-18e4-4201-a547-e6850c782cea\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq" Jul 23 13:15:14.303462 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.303376 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq" Jul 23 13:15:14.434940 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.434907 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq"] Jul 23 13:15:14.437791 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:15:14.437754 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2bae3f4_18e4_4201_a547_e6850c782cea.slice/crio-d330f39523922f5a0bb839b67afedcb3d95b74b668f189044d209adf28a507f8 WatchSource:0}: Error finding container d330f39523922f5a0bb839b67afedcb3d95b74b668f189044d209adf28a507f8: Status 404 returned error can't find the container with id d330f39523922f5a0bb839b67afedcb3d95b74b668f189044d209adf28a507f8 Jul 23 13:15:14.534518 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.534487 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq" event={"ID":"b2bae3f4-18e4-4201-a547-e6850c782cea","Type":"ContainerStarted","Data":"1a46c2802e55dfdccbfb5c1a4c5057e97883e9332a47677e9cf3970c36e9757c"} Jul 23 13:15:14.534660 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.534540 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq" event={"ID":"b2bae3f4-18e4-4201-a547-e6850c782cea","Type":"ContainerStarted","Data":"d330f39523922f5a0bb839b67afedcb3d95b74b668f189044d209adf28a507f8"} Jul 23 13:15:14.589536 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.589505 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr"] Jul 23 13:15:14.592029 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.592014 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr" Jul 23 13:15:14.601497 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.601467 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr"] Jul 23 13:15:14.678016 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.677979 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr\" (UID: \"6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr" Jul 23 13:15:14.678016 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.678021 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr\" (UID: \"6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr" Jul 23 13:15:14.678214 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.678038 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp684\" (UniqueName: \"kubernetes.io/projected/6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024-kube-api-access-hp684\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr\" (UID: \"6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr" Jul 23 13:15:14.779265 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.779225 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr\" (UID: \"6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr" Jul 23 13:15:14.779265 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.779266 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr\" (UID: \"6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr" Jul 23 13:15:14.779497 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.779290 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp684\" (UniqueName: \"kubernetes.io/projected/6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024-kube-api-access-hp684\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr\" (UID: \"6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr" Jul 23 13:15:14.779687 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.779667 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr\" (UID: \"6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr" Jul 23 13:15:14.779725 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.779670 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr\" (UID: \"6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr" Jul 23 13:15:14.787867 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.787838 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp684\" (UniqueName: \"kubernetes.io/projected/6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024-kube-api-access-hp684\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr\" (UID: \"6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr" Jul 23 13:15:14.901891 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.901789 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr" Jul 23 13:15:14.987911 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.987651 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7"] Jul 23 13:15:14.991621 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:14.991599 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7" Jul 23 13:15:15.000892 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.000865 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7"] Jul 23 13:15:15.038597 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.038571 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr"] Jul 23 13:15:15.040237 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:15:15.040205 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d9cc4d4_f4b9_4ac9_84f0_d7eedc1c4024.slice/crio-7c5957c7a2916af36189f18e8a5a22884a7bb64e6fd7d8f65d63762aa3ecd30f WatchSource:0}: Error finding container 7c5957c7a2916af36189f18e8a5a22884a7bb64e6fd7d8f65d63762aa3ecd30f: Status 404 returned error can't find the container with id 7c5957c7a2916af36189f18e8a5a22884a7bb64e6fd7d8f65d63762aa3ecd30f Jul 23 13:15:15.081525 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.081490 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ccln\" (UniqueName: \"kubernetes.io/projected/c0547351-e583-46d8-a4ba-cf61a7b72075-kube-api-access-5ccln\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7\" (UID: \"c0547351-e583-46d8-a4ba-cf61a7b72075\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7" Jul 23 13:15:15.081673 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.081560 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0547351-e583-46d8-a4ba-cf61a7b72075-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7\" (UID: \"c0547351-e583-46d8-a4ba-cf61a7b72075\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7" Jul 23 13:15:15.081673 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.081592 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0547351-e583-46d8-a4ba-cf61a7b72075-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7\" (UID: \"c0547351-e583-46d8-a4ba-cf61a7b72075\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7" Jul 23 13:15:15.182541 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.182513 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0547351-e583-46d8-a4ba-cf61a7b72075-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7\" (UID: \"c0547351-e583-46d8-a4ba-cf61a7b72075\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7" Jul 23 13:15:15.182690 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.182550 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0547351-e583-46d8-a4ba-cf61a7b72075-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7\" (UID: \"c0547351-e583-46d8-a4ba-cf61a7b72075\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7" Jul 23 13:15:15.182690 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.182635 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ccln\" (UniqueName: \"kubernetes.io/projected/c0547351-e583-46d8-a4ba-cf61a7b72075-kube-api-access-5ccln\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7\" (UID: \"c0547351-e583-46d8-a4ba-cf61a7b72075\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7" Jul 23 13:15:15.182964 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.182938 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0547351-e583-46d8-a4ba-cf61a7b72075-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7\" (UID: \"c0547351-e583-46d8-a4ba-cf61a7b72075\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7" Jul 23 13:15:15.183000 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.182964 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0547351-e583-46d8-a4ba-cf61a7b72075-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7\" (UID: \"c0547351-e583-46d8-a4ba-cf61a7b72075\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7" Jul 23 13:15:15.191138 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.191107 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ccln\" (UniqueName: \"kubernetes.io/projected/c0547351-e583-46d8-a4ba-cf61a7b72075-kube-api-access-5ccln\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7\" (UID: \"c0547351-e583-46d8-a4ba-cf61a7b72075\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7" Jul 23 13:15:15.304128 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.304087 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7" Jul 23 13:15:15.436644 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.436620 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7"] Jul 23 13:15:15.438630 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:15:15.438594 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0547351_e583_46d8_a4ba_cf61a7b72075.slice/crio-85fc3c353391cc491ba3b5b20eba27d3bbc32ad3b6d035cf2e6bf50b650adebf WatchSource:0}: Error finding container 85fc3c353391cc491ba3b5b20eba27d3bbc32ad3b6d035cf2e6bf50b650adebf: Status 404 returned error can't find the container with id 85fc3c353391cc491ba3b5b20eba27d3bbc32ad3b6d035cf2e6bf50b650adebf Jul 23 13:15:15.541461 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.541420 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7" event={"ID":"c0547351-e583-46d8-a4ba-cf61a7b72075","Type":"ContainerStarted","Data":"ab5977eea1a63ccc8addf3c5803d7af33ad9ffce4d210fc015f63d65e1249037"} Jul 23 13:15:15.541658 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.541480 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7" event={"ID":"c0547351-e583-46d8-a4ba-cf61a7b72075","Type":"ContainerStarted","Data":"85fc3c353391cc491ba3b5b20eba27d3bbc32ad3b6d035cf2e6bf50b650adebf"} Jul 23 13:15:15.542989 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.542960 2531 generic.go:356] "Generic (PLEG): container finished" podID="6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024" containerID="fc444b235bb810827762001f08c479bc1a81a5101ca3004ef3265d364059341d" exitCode=0 Jul 23 13:15:15.543099 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.543043 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr" event={"ID":"6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024","Type":"ContainerDied","Data":"fc444b235bb810827762001f08c479bc1a81a5101ca3004ef3265d364059341d"} Jul 23 13:15:15.543158 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.543093 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr" event={"ID":"6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024","Type":"ContainerStarted","Data":"7c5957c7a2916af36189f18e8a5a22884a7bb64e6fd7d8f65d63762aa3ecd30f"} Jul 23 13:15:15.544867 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.544838 2531 generic.go:356] "Generic (PLEG): container finished" podID="b2bae3f4-18e4-4201-a547-e6850c782cea" containerID="1a46c2802e55dfdccbfb5c1a4c5057e97883e9332a47677e9cf3970c36e9757c" exitCode=0 Jul 23 13:15:15.544978 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.544907 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq" event={"ID":"b2bae3f4-18e4-4201-a547-e6850c782cea","Type":"ContainerDied","Data":"1a46c2802e55dfdccbfb5c1a4c5057e97883e9332a47677e9cf3970c36e9757c"} Jul 23 13:15:15.591917 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.591885 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782"] Jul 23 13:15:15.594501 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.594474 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782" Jul 23 13:15:15.605297 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.605274 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782"] Jul 23 13:15:15.685889 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.685852 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0da28128-6818-40bd-92a0-9d8d34b6d9e9-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782\" (UID: \"0da28128-6818-40bd-92a0-9d8d34b6d9e9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782" Jul 23 13:15:15.686099 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.685907 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0da28128-6818-40bd-92a0-9d8d34b6d9e9-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782\" (UID: \"0da28128-6818-40bd-92a0-9d8d34b6d9e9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782" Jul 23 13:15:15.686099 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.686060 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tptc8\" (UniqueName: \"kubernetes.io/projected/0da28128-6818-40bd-92a0-9d8d34b6d9e9-kube-api-access-tptc8\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782\" (UID: \"0da28128-6818-40bd-92a0-9d8d34b6d9e9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782" Jul 23 13:15:15.786627 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.786589 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tptc8\" (UniqueName: \"kubernetes.io/projected/0da28128-6818-40bd-92a0-9d8d34b6d9e9-kube-api-access-tptc8\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782\" (UID: \"0da28128-6818-40bd-92a0-9d8d34b6d9e9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782" Jul 23 13:15:15.786796 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.786661 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0da28128-6818-40bd-92a0-9d8d34b6d9e9-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782\" (UID: \"0da28128-6818-40bd-92a0-9d8d34b6d9e9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782" Jul 23 13:15:15.786796 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.786690 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0da28128-6818-40bd-92a0-9d8d34b6d9e9-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782\" (UID: \"0da28128-6818-40bd-92a0-9d8d34b6d9e9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782" Jul 23 13:15:15.787121 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.787101 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0da28128-6818-40bd-92a0-9d8d34b6d9e9-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782\" (UID: \"0da28128-6818-40bd-92a0-9d8d34b6d9e9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782" Jul 23 13:15:15.787160 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.787117 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0da28128-6818-40bd-92a0-9d8d34b6d9e9-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782\" (UID: \"0da28128-6818-40bd-92a0-9d8d34b6d9e9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782" Jul 23 13:15:15.796219 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.796190 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tptc8\" (UniqueName: \"kubernetes.io/projected/0da28128-6818-40bd-92a0-9d8d34b6d9e9-kube-api-access-tptc8\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782\" (UID: \"0da28128-6818-40bd-92a0-9d8d34b6d9e9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782" Jul 23 13:15:15.926491 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:15.926457 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782" Jul 23 13:15:16.057984 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:16.057952 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782"] Jul 23 13:15:16.061799 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:15:16.061774 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0da28128_6818_40bd_92a0_9d8d34b6d9e9.slice/crio-ad45a73ce8c9fdfca20a1d771b2c24df674b23de9eb3d0f0ade9f66a77af32ab WatchSource:0}: Error finding container ad45a73ce8c9fdfca20a1d771b2c24df674b23de9eb3d0f0ade9f66a77af32ab: Status 404 returned error can't find the container with id ad45a73ce8c9fdfca20a1d771b2c24df674b23de9eb3d0f0ade9f66a77af32ab Jul 23 13:15:16.550751 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:16.550670 2531 generic.go:356] "Generic (PLEG): container finished" podID="0da28128-6818-40bd-92a0-9d8d34b6d9e9" containerID="f708f040391c14de4093c0d8115c000a56db3ae5ebddd45e0dae88df18fd0b0c" exitCode=0 Jul 23 13:15:16.550906 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:16.550755 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782" event={"ID":"0da28128-6818-40bd-92a0-9d8d34b6d9e9","Type":"ContainerDied","Data":"f708f040391c14de4093c0d8115c000a56db3ae5ebddd45e0dae88df18fd0b0c"} Jul 23 13:15:16.550906 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:16.550805 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782" event={"ID":"0da28128-6818-40bd-92a0-9d8d34b6d9e9","Type":"ContainerStarted","Data":"ad45a73ce8c9fdfca20a1d771b2c24df674b23de9eb3d0f0ade9f66a77af32ab"} Jul 23 13:15:16.552306 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:16.552288 2531 generic.go:356] "Generic (PLEG): container finished" podID="c0547351-e583-46d8-a4ba-cf61a7b72075" containerID="ab5977eea1a63ccc8addf3c5803d7af33ad9ffce4d210fc015f63d65e1249037" exitCode=0 Jul 23 13:15:16.552416 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:16.552371 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7" event={"ID":"c0547351-e583-46d8-a4ba-cf61a7b72075","Type":"ContainerDied","Data":"ab5977eea1a63ccc8addf3c5803d7af33ad9ffce4d210fc015f63d65e1249037"} Jul 23 13:15:16.554423 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:16.554337 2531 generic.go:356] "Generic (PLEG): container finished" podID="6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024" containerID="a190a3750d0b584a851447681b3291ebecf06197f270aad0f3bc35f4a22fb77c" exitCode=0 Jul 23 13:15:16.554423 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:16.554399 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr" event={"ID":"6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024","Type":"ContainerDied","Data":"a190a3750d0b584a851447681b3291ebecf06197f270aad0f3bc35f4a22fb77c"} Jul 23 13:15:16.556551 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:16.556523 2531 generic.go:356] "Generic (PLEG): container finished" podID="b2bae3f4-18e4-4201-a547-e6850c782cea" containerID="94841f53b0a397d20d4256e8b65cd7ba339a71aeee82aa516770bbb20af9237c" exitCode=0 Jul 23 13:15:16.556656 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:16.556606 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq" event={"ID":"b2bae3f4-18e4-4201-a547-e6850c782cea","Type":"ContainerDied","Data":"94841f53b0a397d20d4256e8b65cd7ba339a71aeee82aa516770bbb20af9237c"} Jul 23 13:15:17.563449 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:17.563412 2531 generic.go:356] "Generic (PLEG): container finished" podID="6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024" containerID="050f056fbb4fee20ae402a3909656107e54a72690761ea1bc6a18ab3a153cfbe" exitCode=0 Jul 23 13:15:17.563930 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:17.563500 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr" event={"ID":"6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024","Type":"ContainerDied","Data":"050f056fbb4fee20ae402a3909656107e54a72690761ea1bc6a18ab3a153cfbe"} Jul 23 13:15:17.565484 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:17.565461 2531 generic.go:356] "Generic (PLEG): container finished" podID="b2bae3f4-18e4-4201-a547-e6850c782cea" containerID="142e7dcf313c6f5e102312d50034cd766c85ec39aedbf68575067f2a4d8ecd04" exitCode=0 Jul 23 13:15:17.565602 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:17.565541 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq" event={"ID":"b2bae3f4-18e4-4201-a547-e6850c782cea","Type":"ContainerDied","Data":"142e7dcf313c6f5e102312d50034cd766c85ec39aedbf68575067f2a4d8ecd04"} Jul 23 13:15:18.572098 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:18.572066 2531 generic.go:356] "Generic (PLEG): container finished" podID="0da28128-6818-40bd-92a0-9d8d34b6d9e9" containerID="f24d2e21f99be96b4c7afeb44bde67aaaaae7d01554b2b23de6066e42dafc89c" exitCode=0 Jul 23 13:15:18.572509 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:18.572160 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782" event={"ID":"0da28128-6818-40bd-92a0-9d8d34b6d9e9","Type":"ContainerDied","Data":"f24d2e21f99be96b4c7afeb44bde67aaaaae7d01554b2b23de6066e42dafc89c"} Jul 23 13:15:18.767590 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:18.767566 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr" Jul 23 13:15:18.770599 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:18.770580 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq" Jul 23 13:15:18.914595 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:18.914521 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b2bae3f4-18e4-4201-a547-e6850c782cea-bundle\") pod \"b2bae3f4-18e4-4201-a547-e6850c782cea\" (UID: \"b2bae3f4-18e4-4201-a547-e6850c782cea\") " Jul 23 13:15:18.914595 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:18.914580 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfmft\" (UniqueName: \"kubernetes.io/projected/b2bae3f4-18e4-4201-a547-e6850c782cea-kube-api-access-pfmft\") pod \"b2bae3f4-18e4-4201-a547-e6850c782cea\" (UID: \"b2bae3f4-18e4-4201-a547-e6850c782cea\") " Jul 23 13:15:18.914791 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:18.914652 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hp684\" (UniqueName: \"kubernetes.io/projected/6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024-kube-api-access-hp684\") pod \"6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024\" (UID: \"6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024\") " Jul 23 13:15:18.914791 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:18.914671 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b2bae3f4-18e4-4201-a547-e6850c782cea-util\") pod \"b2bae3f4-18e4-4201-a547-e6850c782cea\" (UID: \"b2bae3f4-18e4-4201-a547-e6850c782cea\") " Jul 23 13:15:18.914791 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:18.914699 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024-util\") pod \"6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024\" (UID: \"6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024\") " Jul 23 13:15:18.914791 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:18.914733 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024-bundle\") pod \"6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024\" (UID: \"6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024\") " Jul 23 13:15:18.915403 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:18.915337 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2bae3f4-18e4-4201-a547-e6850c782cea-bundle" (OuterVolumeSpecName: "bundle") pod "b2bae3f4-18e4-4201-a547-e6850c782cea" (UID: "b2bae3f4-18e4-4201-a547-e6850c782cea"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:15:18.915634 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:18.915604 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024-bundle" (OuterVolumeSpecName: "bundle") pod "6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024" (UID: "6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:15:18.916922 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:18.916891 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2bae3f4-18e4-4201-a547-e6850c782cea-kube-api-access-pfmft" (OuterVolumeSpecName: "kube-api-access-pfmft") pod "b2bae3f4-18e4-4201-a547-e6850c782cea" (UID: "b2bae3f4-18e4-4201-a547-e6850c782cea"). InnerVolumeSpecName "kube-api-access-pfmft". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:15:18.917291 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:18.917265 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024-kube-api-access-hp684" (OuterVolumeSpecName: "kube-api-access-hp684") pod "6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024" (UID: "6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024"). InnerVolumeSpecName "kube-api-access-hp684". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:15:18.920721 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:18.920688 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024-util" (OuterVolumeSpecName: "util") pod "6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024" (UID: "6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:15:18.920863 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:18.920842 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2bae3f4-18e4-4201-a547-e6850c782cea-util" (OuterVolumeSpecName: "util") pod "b2bae3f4-18e4-4201-a547-e6850c782cea" (UID: "b2bae3f4-18e4-4201-a547-e6850c782cea"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:15:19.015799 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:19.015768 2531 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024-bundle\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:15:19.015799 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:19.015801 2531 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b2bae3f4-18e4-4201-a547-e6850c782cea-bundle\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:15:19.016013 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:19.015817 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfmft\" (UniqueName: \"kubernetes.io/projected/b2bae3f4-18e4-4201-a547-e6850c782cea-kube-api-access-pfmft\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:15:19.016013 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:19.015832 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hp684\" (UniqueName: \"kubernetes.io/projected/6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024-kube-api-access-hp684\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:15:19.016013 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:19.015845 2531 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b2bae3f4-18e4-4201-a547-e6850c782cea-util\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:15:19.016013 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:19.015857 2531 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024-util\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:15:19.580241 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:19.580206 2531 generic.go:356] "Generic (PLEG): container finished" podID="0da28128-6818-40bd-92a0-9d8d34b6d9e9" containerID="d0eb842e5a32b6beaf1108ad49d9838c87d6d6de878bff03c45a9673b6652a42" exitCode=0 Jul 23 13:15:19.580690 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:19.580294 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782" event={"ID":"0da28128-6818-40bd-92a0-9d8d34b6d9e9","Type":"ContainerDied","Data":"d0eb842e5a32b6beaf1108ad49d9838c87d6d6de878bff03c45a9673b6652a42"} Jul 23 13:15:19.582124 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:19.582101 2531 generic.go:356] "Generic (PLEG): container finished" podID="c0547351-e583-46d8-a4ba-cf61a7b72075" containerID="10d3afc2c02799e4a5b38b1237a43dd4e0f63b309232e1537a96c8db852e6e14" exitCode=0 Jul 23 13:15:19.582233 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:19.582168 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7" event={"ID":"c0547351-e583-46d8-a4ba-cf61a7b72075","Type":"ContainerDied","Data":"10d3afc2c02799e4a5b38b1237a43dd4e0f63b309232e1537a96c8db852e6e14"} Jul 23 13:15:19.584267 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:19.584249 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr" Jul 23 13:15:19.584267 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:19.584289 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr" event={"ID":"6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024","Type":"ContainerDied","Data":"7c5957c7a2916af36189f18e8a5a22884a7bb64e6fd7d8f65d63762aa3ecd30f"} Jul 23 13:15:19.584481 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:19.584328 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7c5957c7a2916af36189f18e8a5a22884a7bb64e6fd7d8f65d63762aa3ecd30f" Jul 23 13:15:19.586221 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:19.586202 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq" event={"ID":"b2bae3f4-18e4-4201-a547-e6850c782cea","Type":"ContainerDied","Data":"d330f39523922f5a0bb839b67afedcb3d95b74b668f189044d209adf28a507f8"} Jul 23 13:15:19.586221 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:19.586223 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq" Jul 23 13:15:19.586221 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:19.586233 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d330f39523922f5a0bb839b67afedcb3d95b74b668f189044d209adf28a507f8" Jul 23 13:15:20.599617 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:20.599581 2531 generic.go:356] "Generic (PLEG): container finished" podID="c0547351-e583-46d8-a4ba-cf61a7b72075" containerID="53cd0bdde837ead6460e9d992d5d2ca0de38a72d9a797f537cc39d7246050bff" exitCode=0 Jul 23 13:15:20.600076 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:20.599713 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7" event={"ID":"c0547351-e583-46d8-a4ba-cf61a7b72075","Type":"ContainerDied","Data":"53cd0bdde837ead6460e9d992d5d2ca0de38a72d9a797f537cc39d7246050bff"} Jul 23 13:15:20.727099 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:20.727075 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782" Jul 23 13:15:20.829395 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:20.829335 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0da28128-6818-40bd-92a0-9d8d34b6d9e9-util\") pod \"0da28128-6818-40bd-92a0-9d8d34b6d9e9\" (UID: \"0da28128-6818-40bd-92a0-9d8d34b6d9e9\") " Jul 23 13:15:20.829577 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:20.829430 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0da28128-6818-40bd-92a0-9d8d34b6d9e9-bundle\") pod \"0da28128-6818-40bd-92a0-9d8d34b6d9e9\" (UID: \"0da28128-6818-40bd-92a0-9d8d34b6d9e9\") " Jul 23 13:15:20.829577 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:20.829482 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tptc8\" (UniqueName: \"kubernetes.io/projected/0da28128-6818-40bd-92a0-9d8d34b6d9e9-kube-api-access-tptc8\") pod \"0da28128-6818-40bd-92a0-9d8d34b6d9e9\" (UID: \"0da28128-6818-40bd-92a0-9d8d34b6d9e9\") " Jul 23 13:15:20.829865 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:20.829842 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da28128-6818-40bd-92a0-9d8d34b6d9e9-bundle" (OuterVolumeSpecName: "bundle") pod "0da28128-6818-40bd-92a0-9d8d34b6d9e9" (UID: "0da28128-6818-40bd-92a0-9d8d34b6d9e9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:15:20.831715 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:20.831691 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0da28128-6818-40bd-92a0-9d8d34b6d9e9-kube-api-access-tptc8" (OuterVolumeSpecName: "kube-api-access-tptc8") pod "0da28128-6818-40bd-92a0-9d8d34b6d9e9" (UID: "0da28128-6818-40bd-92a0-9d8d34b6d9e9"). InnerVolumeSpecName "kube-api-access-tptc8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:15:20.835000 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:20.834976 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da28128-6818-40bd-92a0-9d8d34b6d9e9-util" (OuterVolumeSpecName: "util") pod "0da28128-6818-40bd-92a0-9d8d34b6d9e9" (UID: "0da28128-6818-40bd-92a0-9d8d34b6d9e9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:15:20.930421 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:20.930282 2531 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0da28128-6818-40bd-92a0-9d8d34b6d9e9-bundle\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:15:20.930421 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:20.930331 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tptc8\" (UniqueName: \"kubernetes.io/projected/0da28128-6818-40bd-92a0-9d8d34b6d9e9-kube-api-access-tptc8\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:15:20.930421 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:20.930348 2531 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0da28128-6818-40bd-92a0-9d8d34b6d9e9-util\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:15:21.607234 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:21.607153 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782" Jul 23 13:15:21.607234 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:21.607172 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782" event={"ID":"0da28128-6818-40bd-92a0-9d8d34b6d9e9","Type":"ContainerDied","Data":"ad45a73ce8c9fdfca20a1d771b2c24df674b23de9eb3d0f0ade9f66a77af32ab"} Jul 23 13:15:21.607234 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:21.607218 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ad45a73ce8c9fdfca20a1d771b2c24df674b23de9eb3d0f0ade9f66a77af32ab" Jul 23 13:15:21.734728 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:21.734701 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7" Jul 23 13:15:21.838982 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:21.838950 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ccln\" (UniqueName: \"kubernetes.io/projected/c0547351-e583-46d8-a4ba-cf61a7b72075-kube-api-access-5ccln\") pod \"c0547351-e583-46d8-a4ba-cf61a7b72075\" (UID: \"c0547351-e583-46d8-a4ba-cf61a7b72075\") " Jul 23 13:15:21.839156 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:21.839026 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0547351-e583-46d8-a4ba-cf61a7b72075-util\") pod \"c0547351-e583-46d8-a4ba-cf61a7b72075\" (UID: \"c0547351-e583-46d8-a4ba-cf61a7b72075\") " Jul 23 13:15:21.839156 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:21.839106 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0547351-e583-46d8-a4ba-cf61a7b72075-bundle\") pod \"c0547351-e583-46d8-a4ba-cf61a7b72075\" (UID: \"c0547351-e583-46d8-a4ba-cf61a7b72075\") " Jul 23 13:15:21.839662 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:21.839629 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0547351-e583-46d8-a4ba-cf61a7b72075-bundle" (OuterVolumeSpecName: "bundle") pod "c0547351-e583-46d8-a4ba-cf61a7b72075" (UID: "c0547351-e583-46d8-a4ba-cf61a7b72075"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:15:21.841138 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:21.841109 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0547351-e583-46d8-a4ba-cf61a7b72075-kube-api-access-5ccln" (OuterVolumeSpecName: "kube-api-access-5ccln") pod "c0547351-e583-46d8-a4ba-cf61a7b72075" (UID: "c0547351-e583-46d8-a4ba-cf61a7b72075"). InnerVolumeSpecName "kube-api-access-5ccln". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:15:21.843248 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:21.843221 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0547351-e583-46d8-a4ba-cf61a7b72075-util" (OuterVolumeSpecName: "util") pod "c0547351-e583-46d8-a4ba-cf61a7b72075" (UID: "c0547351-e583-46d8-a4ba-cf61a7b72075"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:15:21.940802 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:21.940706 2531 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0547351-e583-46d8-a4ba-cf61a7b72075-bundle\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:15:21.940802 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:21.940748 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ccln\" (UniqueName: \"kubernetes.io/projected/c0547351-e583-46d8-a4ba-cf61a7b72075-kube-api-access-5ccln\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:15:21.940802 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:21.940758 2531 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0547351-e583-46d8-a4ba-cf61a7b72075-util\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:15:22.614478 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:22.614449 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7" Jul 23 13:15:22.614842 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:22.614487 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7" event={"ID":"c0547351-e583-46d8-a4ba-cf61a7b72075","Type":"ContainerDied","Data":"85fc3c353391cc491ba3b5b20eba27d3bbc32ad3b6d035cf2e6bf50b650adebf"} Jul 23 13:15:22.614842 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:22.614533 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="85fc3c353391cc491ba3b5b20eba27d3bbc32ad3b6d035cf2e6bf50b650adebf" Jul 23 13:15:23.346141 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346107 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-96945b589-8n98g"] Jul 23 13:15:23.346473 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346461 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0547351-e583-46d8-a4ba-cf61a7b72075" containerName="extract" Jul 23 13:15:23.346533 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346475 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0547351-e583-46d8-a4ba-cf61a7b72075" containerName="extract" Jul 23 13:15:23.346533 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346486 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0da28128-6818-40bd-92a0-9d8d34b6d9e9" containerName="util" Jul 23 13:15:23.346533 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346492 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="0da28128-6818-40bd-92a0-9d8d34b6d9e9" containerName="util" Jul 23 13:15:23.346533 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346498 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2bae3f4-18e4-4201-a547-e6850c782cea" containerName="pull" Jul 23 13:15:23.346533 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346503 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2bae3f4-18e4-4201-a547-e6850c782cea" containerName="pull" Jul 23 13:15:23.346533 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346512 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024" containerName="extract" Jul 23 13:15:23.346533 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346517 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024" containerName="extract" Jul 23 13:15:23.346533 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346523 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2bae3f4-18e4-4201-a547-e6850c782cea" containerName="util" Jul 23 13:15:23.346533 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346528 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2bae3f4-18e4-4201-a547-e6850c782cea" containerName="util" Jul 23 13:15:23.346533 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346535 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0da28128-6818-40bd-92a0-9d8d34b6d9e9" containerName="extract" Jul 23 13:15:23.346859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346540 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="0da28128-6818-40bd-92a0-9d8d34b6d9e9" containerName="extract" Jul 23 13:15:23.346859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346552 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024" containerName="util" Jul 23 13:15:23.346859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346556 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024" containerName="util" Jul 23 13:15:23.346859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346562 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0547351-e583-46d8-a4ba-cf61a7b72075" containerName="util" Jul 23 13:15:23.346859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346568 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0547351-e583-46d8-a4ba-cf61a7b72075" containerName="util" Jul 23 13:15:23.346859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346573 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2bae3f4-18e4-4201-a547-e6850c782cea" containerName="extract" Jul 23 13:15:23.346859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346578 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2bae3f4-18e4-4201-a547-e6850c782cea" containerName="extract" Jul 23 13:15:23.346859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346585 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0547351-e583-46d8-a4ba-cf61a7b72075" containerName="pull" Jul 23 13:15:23.346859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346590 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0547351-e583-46d8-a4ba-cf61a7b72075" containerName="pull" Jul 23 13:15:23.346859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346596 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0da28128-6818-40bd-92a0-9d8d34b6d9e9" containerName="pull" Jul 23 13:15:23.346859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346600 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="0da28128-6818-40bd-92a0-9d8d34b6d9e9" containerName="pull" Jul 23 13:15:23.346859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346605 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024" containerName="pull" Jul 23 13:15:23.346859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346611 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024" containerName="pull" Jul 23 13:15:23.346859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346659 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0547351-e583-46d8-a4ba-cf61a7b72075" containerName="extract" Jul 23 13:15:23.346859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346671 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2bae3f4-18e4-4201-a547-e6850c782cea" containerName="extract" Jul 23 13:15:23.346859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346676 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024" containerName="extract" Jul 23 13:15:23.346859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.346680 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="0da28128-6818-40bd-92a0-9d8d34b6d9e9" containerName="extract" Jul 23 13:15:23.348908 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.348891 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:23.361891 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.361866 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-96945b589-8n98g"] Jul 23 13:15:23.454051 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.454002 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62544614-8aad-4ff5-9908-4a58860c5f75-console-serving-cert\") pod \"console-96945b589-8n98g\" (UID: \"62544614-8aad-4ff5-9908-4a58860c5f75\") " pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:23.454208 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.454082 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62544614-8aad-4ff5-9908-4a58860c5f75-console-oauth-config\") pod \"console-96945b589-8n98g\" (UID: \"62544614-8aad-4ff5-9908-4a58860c5f75\") " pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:23.454208 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.454103 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62544614-8aad-4ff5-9908-4a58860c5f75-oauth-serving-cert\") pod \"console-96945b589-8n98g\" (UID: \"62544614-8aad-4ff5-9908-4a58860c5f75\") " pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:23.454208 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.454119 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncb6n\" (UniqueName: \"kubernetes.io/projected/62544614-8aad-4ff5-9908-4a58860c5f75-kube-api-access-ncb6n\") pod \"console-96945b589-8n98g\" (UID: \"62544614-8aad-4ff5-9908-4a58860c5f75\") " pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:23.454208 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.454158 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62544614-8aad-4ff5-9908-4a58860c5f75-console-config\") pod \"console-96945b589-8n98g\" (UID: \"62544614-8aad-4ff5-9908-4a58860c5f75\") " pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:23.454393 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.454216 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62544614-8aad-4ff5-9908-4a58860c5f75-service-ca\") pod \"console-96945b589-8n98g\" (UID: \"62544614-8aad-4ff5-9908-4a58860c5f75\") " pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:23.454393 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.454233 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62544614-8aad-4ff5-9908-4a58860c5f75-trusted-ca-bundle\") pod \"console-96945b589-8n98g\" (UID: \"62544614-8aad-4ff5-9908-4a58860c5f75\") " pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:23.555607 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.555559 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62544614-8aad-4ff5-9908-4a58860c5f75-console-oauth-config\") pod \"console-96945b589-8n98g\" (UID: \"62544614-8aad-4ff5-9908-4a58860c5f75\") " pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:23.555607 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.555613 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62544614-8aad-4ff5-9908-4a58860c5f75-oauth-serving-cert\") pod \"console-96945b589-8n98g\" (UID: \"62544614-8aad-4ff5-9908-4a58860c5f75\") " pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:23.555825 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.555632 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncb6n\" (UniqueName: \"kubernetes.io/projected/62544614-8aad-4ff5-9908-4a58860c5f75-kube-api-access-ncb6n\") pod \"console-96945b589-8n98g\" (UID: \"62544614-8aad-4ff5-9908-4a58860c5f75\") " pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:23.555825 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.555665 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62544614-8aad-4ff5-9908-4a58860c5f75-console-config\") pod \"console-96945b589-8n98g\" (UID: \"62544614-8aad-4ff5-9908-4a58860c5f75\") " pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:23.555825 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.555701 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62544614-8aad-4ff5-9908-4a58860c5f75-service-ca\") pod \"console-96945b589-8n98g\" (UID: \"62544614-8aad-4ff5-9908-4a58860c5f75\") " pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:23.555825 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.555716 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62544614-8aad-4ff5-9908-4a58860c5f75-trusted-ca-bundle\") pod \"console-96945b589-8n98g\" (UID: \"62544614-8aad-4ff5-9908-4a58860c5f75\") " pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:23.555825 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.555737 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62544614-8aad-4ff5-9908-4a58860c5f75-console-serving-cert\") pod \"console-96945b589-8n98g\" (UID: \"62544614-8aad-4ff5-9908-4a58860c5f75\") " pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:23.556505 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.556436 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62544614-8aad-4ff5-9908-4a58860c5f75-service-ca\") pod \"console-96945b589-8n98g\" (UID: \"62544614-8aad-4ff5-9908-4a58860c5f75\") " pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:23.556505 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.556485 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62544614-8aad-4ff5-9908-4a58860c5f75-console-config\") pod \"console-96945b589-8n98g\" (UID: \"62544614-8aad-4ff5-9908-4a58860c5f75\") " pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:23.556746 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.556597 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62544614-8aad-4ff5-9908-4a58860c5f75-oauth-serving-cert\") pod \"console-96945b589-8n98g\" (UID: \"62544614-8aad-4ff5-9908-4a58860c5f75\") " pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:23.556746 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.556625 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62544614-8aad-4ff5-9908-4a58860c5f75-trusted-ca-bundle\") pod \"console-96945b589-8n98g\" (UID: \"62544614-8aad-4ff5-9908-4a58860c5f75\") " pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:23.558318 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.558284 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62544614-8aad-4ff5-9908-4a58860c5f75-console-oauth-config\") pod \"console-96945b589-8n98g\" (UID: \"62544614-8aad-4ff5-9908-4a58860c5f75\") " pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:23.558450 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.558321 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62544614-8aad-4ff5-9908-4a58860c5f75-console-serving-cert\") pod \"console-96945b589-8n98g\" (UID: \"62544614-8aad-4ff5-9908-4a58860c5f75\") " pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:23.564685 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.564664 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncb6n\" (UniqueName: \"kubernetes.io/projected/62544614-8aad-4ff5-9908-4a58860c5f75-kube-api-access-ncb6n\") pod \"console-96945b589-8n98g\" (UID: \"62544614-8aad-4ff5-9908-4a58860c5f75\") " pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:23.658799 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.658700 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:23.807500 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:23.807473 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-96945b589-8n98g"] Jul 23 13:15:23.810020 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:15:23.809996 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62544614_8aad_4ff5_9908_4a58860c5f75.slice/crio-7e0e62d2d30ab16e3d7e4e93f39f6d9ca657a2d7209283c5cf5fe1cefbbe2259 WatchSource:0}: Error finding container 7e0e62d2d30ab16e3d7e4e93f39f6d9ca657a2d7209283c5cf5fe1cefbbe2259: Status 404 returned error can't find the container with id 7e0e62d2d30ab16e3d7e4e93f39f6d9ca657a2d7209283c5cf5fe1cefbbe2259 Jul 23 13:15:24.626023 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:24.625988 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-96945b589-8n98g" event={"ID":"62544614-8aad-4ff5-9908-4a58860c5f75","Type":"ContainerStarted","Data":"c597e34e095c9d08d591092b95a7043d1df29f2167ab0d39b325e0167a1ffcbb"} Jul 23 13:15:24.626204 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:24.626038 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-96945b589-8n98g" event={"ID":"62544614-8aad-4ff5-9908-4a58860c5f75","Type":"ContainerStarted","Data":"7e0e62d2d30ab16e3d7e4e93f39f6d9ca657a2d7209283c5cf5fe1cefbbe2259"} Jul 23 13:15:24.644425 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:24.644377 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-96945b589-8n98g" podStartSLOduration=1.644341799 podStartE2EDuration="1.644341799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:15:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:15:24.643290868 +0000 UTC m=+461.882566863" watchObservedRunningTime="2026-07-23 13:15:24.644341799 +0000 UTC m=+461.883617774" Jul 23 13:15:32.372650 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:32.372615 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw"] Jul 23 13:15:32.375308 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:32.375291 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw" Jul 23 13:15:32.378508 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:32.378482 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-q5mc4\"" Jul 23 13:15:32.399230 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:32.399201 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw"] Jul 23 13:15:32.436755 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:32.436724 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mb85\" (UniqueName: \"kubernetes.io/projected/972002d1-318c-4a18-882c-ee7b3d367d20-kube-api-access-7mb85\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t2xkw\" (UID: \"972002d1-318c-4a18-882c-ee7b3d367d20\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw" Jul 23 13:15:32.436906 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:32.436806 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/972002d1-318c-4a18-882c-ee7b3d367d20-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t2xkw\" (UID: \"972002d1-318c-4a18-882c-ee7b3d367d20\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw" Jul 23 13:15:32.537576 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:32.537548 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/972002d1-318c-4a18-882c-ee7b3d367d20-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t2xkw\" (UID: \"972002d1-318c-4a18-882c-ee7b3d367d20\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw" Jul 23 13:15:32.537747 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:32.537642 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mb85\" (UniqueName: \"kubernetes.io/projected/972002d1-318c-4a18-882c-ee7b3d367d20-kube-api-access-7mb85\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t2xkw\" (UID: \"972002d1-318c-4a18-882c-ee7b3d367d20\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw" Jul 23 13:15:32.538009 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:32.537986 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/972002d1-318c-4a18-882c-ee7b3d367d20-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t2xkw\" (UID: \"972002d1-318c-4a18-882c-ee7b3d367d20\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw" Jul 23 13:15:32.546917 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:32.546895 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mb85\" (UniqueName: \"kubernetes.io/projected/972002d1-318c-4a18-882c-ee7b3d367d20-kube-api-access-7mb85\") pod \"kuadrant-operator-controller-manager-8684c9ff49-t2xkw\" (UID: \"972002d1-318c-4a18-882c-ee7b3d367d20\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw" Jul 23 13:15:32.685634 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:32.685549 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw" Jul 23 13:15:32.844233 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:32.844208 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw"] Jul 23 13:15:32.846292 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:15:32.846262 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod972002d1_318c_4a18_882c_ee7b3d367d20.slice/crio-bd9c7a87230deee9140434ba7d88dccb32cf2b5778faba004d2916c514822ca8 WatchSource:0}: Error finding container bd9c7a87230deee9140434ba7d88dccb32cf2b5778faba004d2916c514822ca8: Status 404 returned error can't find the container with id bd9c7a87230deee9140434ba7d88dccb32cf2b5778faba004d2916c514822ca8 Jul 23 13:15:33.658964 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:33.658931 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:33.659413 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:33.658980 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:33.665240 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:33.665217 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:33.673702 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:33.673670 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw" event={"ID":"972002d1-318c-4a18-882c-ee7b3d367d20","Type":"ContainerStarted","Data":"bd9c7a87230deee9140434ba7d88dccb32cf2b5778faba004d2916c514822ca8"} Jul 23 13:15:33.677437 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:33.677420 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-96945b589-8n98g" Jul 23 13:15:33.756795 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:33.756761 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6fff7954c7-48vgr"] Jul 23 13:15:38.704534 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:38.704442 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw" event={"ID":"972002d1-318c-4a18-882c-ee7b3d367d20","Type":"ContainerStarted","Data":"dbb82ec40664e0d588bdbdc1619bac5a75f4c8ac3ab6f30bf4fc171a3df9ed61"} Jul 23 13:15:38.704534 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:38.704528 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw" Jul 23 13:15:38.728131 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:38.728075 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw" podStartSLOduration=1.25755586 podStartE2EDuration="6.728055746s" totalImagesPullingTime="5.470499886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:15:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:15:38.726224769 +0000 UTC m=+475.965500745" watchObservedRunningTime="2026-07-23 13:15:38.728055746 +0000 UTC m=+475.967331722" Jul 23 13:15:49.712479 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:49.712447 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw" Jul 23 13:15:51.683075 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:51.683028 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw"] Jul 23 13:15:51.683553 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:51.683264 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw" podUID="972002d1-318c-4a18-882c-ee7b3d367d20" containerName="manager" containerID="cri-o://dbb82ec40664e0d588bdbdc1619bac5a75f4c8ac3ab6f30bf4fc171a3df9ed61" gracePeriod=2 Jul 23 13:15:51.693789 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:15:51.693745 2531 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"972002d1-318c-4a18-882c-ee7b3d367d20\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-t2xkw\": pods \"kuadrant-operator-controller-manager-8684c9ff49-t2xkw\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw" Jul 23 13:15:51.694402 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:51.694381 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw"] Jul 23 13:15:51.732657 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:51.732621 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qd75"] Jul 23 13:15:51.733214 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:51.733191 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="972002d1-318c-4a18-882c-ee7b3d367d20" containerName="manager" Jul 23 13:15:51.733214 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:51.733216 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="972002d1-318c-4a18-882c-ee7b3d367d20" containerName="manager" Jul 23 13:15:51.733391 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:51.733308 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="972002d1-318c-4a18-882c-ee7b3d367d20" containerName="manager" Jul 23 13:15:51.736958 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:51.736935 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qd75" Jul 23 13:15:51.739462 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:15:51.739435 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-t2xkw\" is forbidden: User \"system:node:ip-10-0-130-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-1.ec2.internal' and this object" podUID="972002d1-318c-4a18-882c-ee7b3d367d20" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw" Jul 23 13:15:51.760443 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:51.760411 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qd75"] Jul 23 13:15:51.773042 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:51.773014 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dkp6n"] Jul 23 13:15:51.776623 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:51.776606 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dkp6n" Jul 23 13:15:51.778947 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:51.778931 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-t5wv8\"" Jul 23 13:15:51.791860 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:15:51.791827 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-t2xkw\" is forbidden: User \"system:node:ip-10-0-130-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-1.ec2.internal' and this object" podUID="972002d1-318c-4a18-882c-ee7b3d367d20" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw" Jul 23 13:15:51.793461 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:51.793436 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dkp6n"] Jul 23 13:15:51.826628 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:51.826595 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/61132ee0-12d1-463e-a2e4-153496ada16d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5qd75\" (UID: \"61132ee0-12d1-463e-a2e4-153496ada16d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qd75" Jul 23 13:15:51.826628 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:51.826632 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7km5d\" (UniqueName: \"kubernetes.io/projected/61132ee0-12d1-463e-a2e4-153496ada16d-kube-api-access-7km5d\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5qd75\" (UID: \"61132ee0-12d1-463e-a2e4-153496ada16d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qd75" Jul 23 13:15:51.927283 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:51.927240 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/61132ee0-12d1-463e-a2e4-153496ada16d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5qd75\" (UID: \"61132ee0-12d1-463e-a2e4-153496ada16d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qd75" Jul 23 13:15:51.927283 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:51.927282 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7km5d\" (UniqueName: \"kubernetes.io/projected/61132ee0-12d1-463e-a2e4-153496ada16d-kube-api-access-7km5d\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5qd75\" (UID: \"61132ee0-12d1-463e-a2e4-153496ada16d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qd75" Jul 23 13:15:51.927501 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:51.927329 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8w5b\" (UniqueName: \"kubernetes.io/projected/0ad71b4e-4c7f-46ae-a26f-1d461c3cfd08-kube-api-access-k8w5b\") pod \"limitador-operator-controller-manager-859d7b78bd-dkp6n\" (UID: \"0ad71b4e-4c7f-46ae-a26f-1d461c3cfd08\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dkp6n" Jul 23 13:15:51.927694 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:51.927670 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/61132ee0-12d1-463e-a2e4-153496ada16d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5qd75\" (UID: \"61132ee0-12d1-463e-a2e4-153496ada16d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qd75" Jul 23 13:15:51.930826 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:51.930804 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw" Jul 23 13:15:51.934426 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:15:51.934377 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-t2xkw\" is forbidden: User \"system:node:ip-10-0-130-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-1.ec2.internal' and this object" podUID="972002d1-318c-4a18-882c-ee7b3d367d20" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw" Jul 23 13:15:51.936889 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:51.936867 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7km5d\" (UniqueName: \"kubernetes.io/projected/61132ee0-12d1-463e-a2e4-153496ada16d-kube-api-access-7km5d\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5qd75\" (UID: \"61132ee0-12d1-463e-a2e4-153496ada16d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qd75" Jul 23 13:15:52.028593 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:52.028561 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/972002d1-318c-4a18-882c-ee7b3d367d20-extensions-socket-volume\") pod \"972002d1-318c-4a18-882c-ee7b3d367d20\" (UID: \"972002d1-318c-4a18-882c-ee7b3d367d20\") " Jul 23 13:15:52.028785 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:52.028670 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mb85\" (UniqueName: \"kubernetes.io/projected/972002d1-318c-4a18-882c-ee7b3d367d20-kube-api-access-7mb85\") pod \"972002d1-318c-4a18-882c-ee7b3d367d20\" (UID: \"972002d1-318c-4a18-882c-ee7b3d367d20\") " Jul 23 13:15:52.028913 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:52.028895 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8w5b\" (UniqueName: \"kubernetes.io/projected/0ad71b4e-4c7f-46ae-a26f-1d461c3cfd08-kube-api-access-k8w5b\") pod \"limitador-operator-controller-manager-859d7b78bd-dkp6n\" (UID: \"0ad71b4e-4c7f-46ae-a26f-1d461c3cfd08\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dkp6n" Jul 23 13:15:52.029029 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:52.029002 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/972002d1-318c-4a18-882c-ee7b3d367d20-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "972002d1-318c-4a18-882c-ee7b3d367d20" (UID: "972002d1-318c-4a18-882c-ee7b3d367d20"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:15:52.030989 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:52.030960 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/972002d1-318c-4a18-882c-ee7b3d367d20-kube-api-access-7mb85" (OuterVolumeSpecName: "kube-api-access-7mb85") pod "972002d1-318c-4a18-882c-ee7b3d367d20" (UID: "972002d1-318c-4a18-882c-ee7b3d367d20"). InnerVolumeSpecName "kube-api-access-7mb85". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:15:52.039986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:52.039964 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8w5b\" (UniqueName: \"kubernetes.io/projected/0ad71b4e-4c7f-46ae-a26f-1d461c3cfd08-kube-api-access-k8w5b\") pod \"limitador-operator-controller-manager-859d7b78bd-dkp6n\" (UID: \"0ad71b4e-4c7f-46ae-a26f-1d461c3cfd08\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dkp6n" Jul 23 13:15:52.066864 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:52.066829 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qd75" Jul 23 13:15:52.086517 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:52.086485 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dkp6n" Jul 23 13:15:52.130347 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:52.130312 2531 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/972002d1-318c-4a18-882c-ee7b3d367d20-extensions-socket-volume\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:15:52.130347 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:52.130379 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mb85\" (UniqueName: \"kubernetes.io/projected/972002d1-318c-4a18-882c-ee7b3d367d20-kube-api-access-7mb85\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:15:52.214187 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:52.214103 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qd75"] Jul 23 13:15:52.216803 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:15:52.216771 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61132ee0_12d1_463e_a2e4_153496ada16d.slice/crio-bb694de1d50248714d6a6704b3c4b2573361e86fe47354bfdb9bbc72bb536a18 WatchSource:0}: Error finding container bb694de1d50248714d6a6704b3c4b2573361e86fe47354bfdb9bbc72bb536a18: Status 404 returned error can't find the container with id bb694de1d50248714d6a6704b3c4b2573361e86fe47354bfdb9bbc72bb536a18 Jul 23 13:15:52.240142 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:52.240115 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dkp6n"] Jul 23 13:15:52.241635 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:15:52.241607 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ad71b4e_4c7f_46ae_a26f_1d461c3cfd08.slice/crio-8f1c9c43d33bb28e2e57349776b7d8f40194c277c36ff8924e58de0fed2a1ec1 WatchSource:0}: Error finding container 8f1c9c43d33bb28e2e57349776b7d8f40194c277c36ff8924e58de0fed2a1ec1: Status 404 returned error can't find the container with id 8f1c9c43d33bb28e2e57349776b7d8f40194c277c36ff8924e58de0fed2a1ec1 Jul 23 13:15:52.785316 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:52.785275 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dkp6n" event={"ID":"0ad71b4e-4c7f-46ae-a26f-1d461c3cfd08","Type":"ContainerStarted","Data":"8f1c9c43d33bb28e2e57349776b7d8f40194c277c36ff8924e58de0fed2a1ec1"} Jul 23 13:15:52.787125 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:52.787091 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qd75" event={"ID":"61132ee0-12d1-463e-a2e4-153496ada16d","Type":"ContainerStarted","Data":"7a96a6a11830e2eaa4997b61898134e07809d7599dbec385e52f1108edf9b2c5"} Jul 23 13:15:52.787382 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:52.787152 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qd75" event={"ID":"61132ee0-12d1-463e-a2e4-153496ada16d","Type":"ContainerStarted","Data":"bb694de1d50248714d6a6704b3c4b2573361e86fe47354bfdb9bbc72bb536a18"} Jul 23 13:15:52.787382 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:52.787255 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qd75" Jul 23 13:15:52.788635 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:52.788607 2531 generic.go:356] "Generic (PLEG): container finished" podID="972002d1-318c-4a18-882c-ee7b3d367d20" containerID="dbb82ec40664e0d588bdbdc1619bac5a75f4c8ac3ab6f30bf4fc171a3df9ed61" exitCode=0 Jul 23 13:15:52.788722 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:52.788649 2531 scope.go:117] "RemoveContainer" containerID="dbb82ec40664e0d588bdbdc1619bac5a75f4c8ac3ab6f30bf4fc171a3df9ed61" Jul 23 13:15:52.788768 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:52.788654 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw" Jul 23 13:15:52.789565 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:15:52.789536 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-t2xkw\" is forbidden: User \"system:node:ip-10-0-130-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-1.ec2.internal' and this object" podUID="972002d1-318c-4a18-882c-ee7b3d367d20" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw" Jul 23 13:15:52.818260 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:15:52.818219 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-t2xkw\" is forbidden: User \"system:node:ip-10-0-130-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-1.ec2.internal' and this object" podUID="972002d1-318c-4a18-882c-ee7b3d367d20" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw" Jul 23 13:15:52.818260 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:52.818205 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qd75" podStartSLOduration=1.818187217 podStartE2EDuration="1.818187217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:15:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:15:52.815746867 +0000 UTC m=+490.055022867" watchObservedRunningTime="2026-07-23 13:15:52.818187217 +0000 UTC m=+490.057463198" Jul 23 13:15:52.827150 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:52.827126 2531 scope.go:117] "RemoveContainer" containerID="dbb82ec40664e0d588bdbdc1619bac5a75f4c8ac3ab6f30bf4fc171a3df9ed61" Jul 23 13:15:52.827542 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:15:52.827515 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbb82ec40664e0d588bdbdc1619bac5a75f4c8ac3ab6f30bf4fc171a3df9ed61\": container with ID starting with dbb82ec40664e0d588bdbdc1619bac5a75f4c8ac3ab6f30bf4fc171a3df9ed61 not found: ID does not exist" containerID="dbb82ec40664e0d588bdbdc1619bac5a75f4c8ac3ab6f30bf4fc171a3df9ed61" Jul 23 13:15:52.827653 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:52.827556 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbb82ec40664e0d588bdbdc1619bac5a75f4c8ac3ab6f30bf4fc171a3df9ed61"} err="failed to get container status \"dbb82ec40664e0d588bdbdc1619bac5a75f4c8ac3ab6f30bf4fc171a3df9ed61\": rpc error: code = NotFound desc = could not find container \"dbb82ec40664e0d588bdbdc1619bac5a75f4c8ac3ab6f30bf4fc171a3df9ed61\": container with ID starting with dbb82ec40664e0d588bdbdc1619bac5a75f4c8ac3ab6f30bf4fc171a3df9ed61 not found: ID does not exist" Jul 23 13:15:53.333592 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:15:53.333547 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-t2xkw\" is forbidden: User \"system:node:ip-10-0-130-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-1.ec2.internal' and this object" podUID="972002d1-318c-4a18-882c-ee7b3d367d20" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-t2xkw" Jul 23 13:15:53.340655 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:53.340624 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="972002d1-318c-4a18-882c-ee7b3d367d20" path="/var/lib/kubelet/pods/972002d1-318c-4a18-882c-ee7b3d367d20/volumes" Jul 23 13:15:54.802981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:54.802945 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dkp6n" event={"ID":"0ad71b4e-4c7f-46ae-a26f-1d461c3cfd08","Type":"ContainerStarted","Data":"ea8c3b10d30f87b14c7a72406099d15934eb7223644fa2c029dd91053958b1e2"} Jul 23 13:15:54.803378 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:54.803011 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dkp6n" Jul 23 13:15:58.789391 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:58.789327 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6fff7954c7-48vgr" podUID="69b9013b-1695-4f0e-bc47-bbd8101610c8" containerName="console" containerID="cri-o://71d630d2795d22c092a524d2e117fbeb0bbef756213c2d19cafcbee047aa312d" gracePeriod=15 Jul 23 13:15:59.040404 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.040324 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:15:59.060028 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.059980 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dkp6n" podStartSLOduration=6.376529024 podStartE2EDuration="8.05996318s" totalImagesPullingTime="1.683434156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:15:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:15:54.847307577 +0000 UTC m=+492.086583554" watchObservedRunningTime="2026-07-23 13:15:59.05996318 +0000 UTC m=+496.299239157" Jul 23 13:15:59.194551 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.194520 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69b9013b-1695-4f0e-bc47-bbd8101610c8-console-oauth-config\") pod \"69b9013b-1695-4f0e-bc47-bbd8101610c8\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " Jul 23 13:15:59.194735 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.194560 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69b9013b-1695-4f0e-bc47-bbd8101610c8-service-ca\") pod \"69b9013b-1695-4f0e-bc47-bbd8101610c8\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " Jul 23 13:15:59.194735 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.194588 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69b9013b-1695-4f0e-bc47-bbd8101610c8-trusted-ca-bundle\") pod \"69b9013b-1695-4f0e-bc47-bbd8101610c8\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " Jul 23 13:15:59.194735 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.194696 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69b9013b-1695-4f0e-bc47-bbd8101610c8-console-serving-cert\") pod \"69b9013b-1695-4f0e-bc47-bbd8101610c8\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " Jul 23 13:15:59.194735 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.194733 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4blf\" (UniqueName: \"kubernetes.io/projected/69b9013b-1695-4f0e-bc47-bbd8101610c8-kube-api-access-s4blf\") pod \"69b9013b-1695-4f0e-bc47-bbd8101610c8\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " Jul 23 13:15:59.194941 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.194780 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69b9013b-1695-4f0e-bc47-bbd8101610c8-oauth-serving-cert\") pod \"69b9013b-1695-4f0e-bc47-bbd8101610c8\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " Jul 23 13:15:59.194941 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.194805 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69b9013b-1695-4f0e-bc47-bbd8101610c8-console-config\") pod \"69b9013b-1695-4f0e-bc47-bbd8101610c8\" (UID: \"69b9013b-1695-4f0e-bc47-bbd8101610c8\") " Jul 23 13:15:59.195099 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.195074 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69b9013b-1695-4f0e-bc47-bbd8101610c8-service-ca" (OuterVolumeSpecName: "service-ca") pod "69b9013b-1695-4f0e-bc47-bbd8101610c8" (UID: "69b9013b-1695-4f0e-bc47-bbd8101610c8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:15:59.195159 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.195108 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69b9013b-1695-4f0e-bc47-bbd8101610c8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "69b9013b-1695-4f0e-bc47-bbd8101610c8" (UID: "69b9013b-1695-4f0e-bc47-bbd8101610c8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:15:59.195407 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.195346 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69b9013b-1695-4f0e-bc47-bbd8101610c8-console-config" (OuterVolumeSpecName: "console-config") pod "69b9013b-1695-4f0e-bc47-bbd8101610c8" (UID: "69b9013b-1695-4f0e-bc47-bbd8101610c8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:15:59.195516 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.195217 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69b9013b-1695-4f0e-bc47-bbd8101610c8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "69b9013b-1695-4f0e-bc47-bbd8101610c8" (UID: "69b9013b-1695-4f0e-bc47-bbd8101610c8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:15:59.196902 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.196881 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69b9013b-1695-4f0e-bc47-bbd8101610c8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "69b9013b-1695-4f0e-bc47-bbd8101610c8" (UID: "69b9013b-1695-4f0e-bc47-bbd8101610c8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 13:15:59.196902 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.196886 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69b9013b-1695-4f0e-bc47-bbd8101610c8-kube-api-access-s4blf" (OuterVolumeSpecName: "kube-api-access-s4blf") pod "69b9013b-1695-4f0e-bc47-bbd8101610c8" (UID: "69b9013b-1695-4f0e-bc47-bbd8101610c8"). InnerVolumeSpecName "kube-api-access-s4blf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:15:59.197037 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.196973 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69b9013b-1695-4f0e-bc47-bbd8101610c8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "69b9013b-1695-4f0e-bc47-bbd8101610c8" (UID: "69b9013b-1695-4f0e-bc47-bbd8101610c8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 13:15:59.296164 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.296075 2531 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69b9013b-1695-4f0e-bc47-bbd8101610c8-console-oauth-config\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:15:59.296164 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.296100 2531 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69b9013b-1695-4f0e-bc47-bbd8101610c8-service-ca\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:15:59.296164 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.296110 2531 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69b9013b-1695-4f0e-bc47-bbd8101610c8-trusted-ca-bundle\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:15:59.296164 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.296118 2531 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69b9013b-1695-4f0e-bc47-bbd8101610c8-console-serving-cert\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:15:59.296164 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.296129 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4blf\" (UniqueName: \"kubernetes.io/projected/69b9013b-1695-4f0e-bc47-bbd8101610c8-kube-api-access-s4blf\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:15:59.296164 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.296137 2531 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69b9013b-1695-4f0e-bc47-bbd8101610c8-oauth-serving-cert\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:15:59.296164 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.296145 2531 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69b9013b-1695-4f0e-bc47-bbd8101610c8-console-config\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:15:59.829884 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.829850 2531 generic.go:356] "Generic (PLEG): container finished" podID="69b9013b-1695-4f0e-bc47-bbd8101610c8" containerID="71d630d2795d22c092a524d2e117fbeb0bbef756213c2d19cafcbee047aa312d" exitCode=0 Jul 23 13:15:59.830315 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.829907 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fff7954c7-48vgr" Jul 23 13:15:59.830315 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.829917 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fff7954c7-48vgr" event={"ID":"69b9013b-1695-4f0e-bc47-bbd8101610c8","Type":"ContainerDied","Data":"71d630d2795d22c092a524d2e117fbeb0bbef756213c2d19cafcbee047aa312d"} Jul 23 13:15:59.830315 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.829958 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fff7954c7-48vgr" event={"ID":"69b9013b-1695-4f0e-bc47-bbd8101610c8","Type":"ContainerDied","Data":"5064daeea0fcedbd8d505201302ff1f7dd23267ad361aa5ea9520a5d6831667e"} Jul 23 13:15:59.830315 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.829976 2531 scope.go:117] "RemoveContainer" containerID="71d630d2795d22c092a524d2e117fbeb0bbef756213c2d19cafcbee047aa312d" Jul 23 13:15:59.839045 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.839028 2531 scope.go:117] "RemoveContainer" containerID="71d630d2795d22c092a524d2e117fbeb0bbef756213c2d19cafcbee047aa312d" Jul 23 13:15:59.839298 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:15:59.839279 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71d630d2795d22c092a524d2e117fbeb0bbef756213c2d19cafcbee047aa312d\": container with ID starting with 71d630d2795d22c092a524d2e117fbeb0bbef756213c2d19cafcbee047aa312d not found: ID does not exist" containerID="71d630d2795d22c092a524d2e117fbeb0bbef756213c2d19cafcbee047aa312d" Jul 23 13:15:59.839388 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.839304 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71d630d2795d22c092a524d2e117fbeb0bbef756213c2d19cafcbee047aa312d"} err="failed to get container status \"71d630d2795d22c092a524d2e117fbeb0bbef756213c2d19cafcbee047aa312d\": rpc error: code = NotFound desc = could not find container \"71d630d2795d22c092a524d2e117fbeb0bbef756213c2d19cafcbee047aa312d\": container with ID starting with 71d630d2795d22c092a524d2e117fbeb0bbef756213c2d19cafcbee047aa312d not found: ID does not exist" Jul 23 13:15:59.847076 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.847053 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6fff7954c7-48vgr"] Jul 23 13:15:59.853679 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:15:59.853652 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6fff7954c7-48vgr"] Jul 23 13:16:01.332135 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:01.332102 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69b9013b-1695-4f0e-bc47-bbd8101610c8" path="/var/lib/kubelet/pods/69b9013b-1695-4f0e-bc47-bbd8101610c8/volumes" Jul 23 13:16:03.798554 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:03.798525 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qd75" Jul 23 13:16:05.811500 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:05.811471 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dkp6n" Jul 23 13:16:20.641337 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:20.641294 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qd75"] Jul 23 13:16:20.641857 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:20.641562 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qd75" podUID="61132ee0-12d1-463e-a2e4-153496ada16d" containerName="manager" containerID="cri-o://7a96a6a11830e2eaa4997b61898134e07809d7599dbec385e52f1108edf9b2c5" gracePeriod=10 Jul 23 13:16:20.890828 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:20.890796 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qd75" Jul 23 13:16:20.944447 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:20.944374 2531 generic.go:356] "Generic (PLEG): container finished" podID="61132ee0-12d1-463e-a2e4-153496ada16d" containerID="7a96a6a11830e2eaa4997b61898134e07809d7599dbec385e52f1108edf9b2c5" exitCode=0 Jul 23 13:16:20.944727 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:20.944476 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qd75" Jul 23 13:16:20.944727 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:20.944491 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qd75" event={"ID":"61132ee0-12d1-463e-a2e4-153496ada16d","Type":"ContainerDied","Data":"7a96a6a11830e2eaa4997b61898134e07809d7599dbec385e52f1108edf9b2c5"} Jul 23 13:16:20.944727 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:20.944533 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qd75" event={"ID":"61132ee0-12d1-463e-a2e4-153496ada16d","Type":"ContainerDied","Data":"bb694de1d50248714d6a6704b3c4b2573361e86fe47354bfdb9bbc72bb536a18"} Jul 23 13:16:20.944727 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:20.944565 2531 scope.go:117] "RemoveContainer" containerID="7a96a6a11830e2eaa4997b61898134e07809d7599dbec385e52f1108edf9b2c5" Jul 23 13:16:20.953910 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:20.953884 2531 scope.go:117] "RemoveContainer" containerID="7a96a6a11830e2eaa4997b61898134e07809d7599dbec385e52f1108edf9b2c5" Jul 23 13:16:20.954174 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:16:20.954154 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a96a6a11830e2eaa4997b61898134e07809d7599dbec385e52f1108edf9b2c5\": container with ID starting with 7a96a6a11830e2eaa4997b61898134e07809d7599dbec385e52f1108edf9b2c5 not found: ID does not exist" containerID="7a96a6a11830e2eaa4997b61898134e07809d7599dbec385e52f1108edf9b2c5" Jul 23 13:16:20.954236 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:20.954182 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a96a6a11830e2eaa4997b61898134e07809d7599dbec385e52f1108edf9b2c5"} err="failed to get container status \"7a96a6a11830e2eaa4997b61898134e07809d7599dbec385e52f1108edf9b2c5\": rpc error: code = NotFound desc = could not find container \"7a96a6a11830e2eaa4997b61898134e07809d7599dbec385e52f1108edf9b2c5\": container with ID starting with 7a96a6a11830e2eaa4997b61898134e07809d7599dbec385e52f1108edf9b2c5 not found: ID does not exist" Jul 23 13:16:20.984643 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:20.984615 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7km5d\" (UniqueName: \"kubernetes.io/projected/61132ee0-12d1-463e-a2e4-153496ada16d-kube-api-access-7km5d\") pod \"61132ee0-12d1-463e-a2e4-153496ada16d\" (UID: \"61132ee0-12d1-463e-a2e4-153496ada16d\") " Jul 23 13:16:20.984768 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:20.984698 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/61132ee0-12d1-463e-a2e4-153496ada16d-extensions-socket-volume\") pod \"61132ee0-12d1-463e-a2e4-153496ada16d\" (UID: \"61132ee0-12d1-463e-a2e4-153496ada16d\") " Jul 23 13:16:20.985064 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:20.985036 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61132ee0-12d1-463e-a2e4-153496ada16d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "61132ee0-12d1-463e-a2e4-153496ada16d" (UID: "61132ee0-12d1-463e-a2e4-153496ada16d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:16:20.986776 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:20.986757 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61132ee0-12d1-463e-a2e4-153496ada16d-kube-api-access-7km5d" (OuterVolumeSpecName: "kube-api-access-7km5d") pod "61132ee0-12d1-463e-a2e4-153496ada16d" (UID: "61132ee0-12d1-463e-a2e4-153496ada16d"). InnerVolumeSpecName "kube-api-access-7km5d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:16:21.085940 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:21.085910 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7km5d\" (UniqueName: \"kubernetes.io/projected/61132ee0-12d1-463e-a2e4-153496ada16d-kube-api-access-7km5d\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:16:21.085940 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:21.085935 2531 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/61132ee0-12d1-463e-a2e4-153496ada16d-extensions-socket-volume\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:16:21.267270 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:21.267243 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qd75"] Jul 23 13:16:21.272015 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:21.271993 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qd75"] Jul 23 13:16:21.332174 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:21.332141 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61132ee0-12d1-463e-a2e4-153496ada16d" path="/var/lib/kubelet/pods/61132ee0-12d1-463e-a2e4-153496ada16d/volumes" Jul 23 13:16:37.021051 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.020982 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp"] Jul 23 13:16:37.021522 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.021425 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69b9013b-1695-4f0e-bc47-bbd8101610c8" containerName="console" Jul 23 13:16:37.021522 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.021439 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="69b9013b-1695-4f0e-bc47-bbd8101610c8" containerName="console" Jul 23 13:16:37.021522 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.021465 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61132ee0-12d1-463e-a2e4-153496ada16d" containerName="manager" Jul 23 13:16:37.021522 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.021470 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="61132ee0-12d1-463e-a2e4-153496ada16d" containerName="manager" Jul 23 13:16:37.021669 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.021526 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="69b9013b-1695-4f0e-bc47-bbd8101610c8" containerName="console" Jul 23 13:16:37.021669 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.021535 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="61132ee0-12d1-463e-a2e4-153496ada16d" containerName="manager" Jul 23 13:16:37.024001 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.023977 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.027719 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.027698 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-kl9pv\"" Jul 23 13:16:37.053977 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.053953 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp"] Jul 23 13:16:37.141604 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.141568 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.141759 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.141630 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.141759 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.141677 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.141759 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.141703 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.141860 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.141758 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.141860 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.141778 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.141860 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.141807 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rh68\" (UniqueName: \"kubernetes.io/projected/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-kube-api-access-4rh68\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.142115 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.141862 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.142115 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.141888 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.243179 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.243123 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.243404 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.243190 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.243404 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.243228 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.243404 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.243259 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.243404 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.243298 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.243404 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.243334 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.243404 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.243391 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.243775 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.243417 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.243775 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.243465 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rh68\" (UniqueName: \"kubernetes.io/projected/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-kube-api-access-4rh68\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.243775 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.243650 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.243985 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.243954 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.244134 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.244108 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.244281 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.244261 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.244626 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.244602 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.246482 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.246461 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.246704 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.246678 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.252690 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.252662 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.252814 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.252797 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rh68\" (UniqueName: \"kubernetes.io/projected/ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2-kube-api-access-4rh68\") pod \"maas-default-gateway-openshift-default-5896d4dd49-lxllp\" (UID: \"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.333737 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.333705 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:37.471732 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.471660 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp"] Jul 23 13:16:37.481902 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:16:37.481871 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff43aeb7_d1b7_425e_94fd_cf95f4e44ed2.slice/crio-187a692ba49644f46fa33cc9855f0274125a858004db834d22b2873a125dfc69 WatchSource:0}: Error finding container 187a692ba49644f46fa33cc9855f0274125a858004db834d22b2873a125dfc69: Status 404 returned error can't find the container with id 187a692ba49644f46fa33cc9855f0274125a858004db834d22b2873a125dfc69 Jul 23 13:16:37.484201 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.484168 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:16:37.484281 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.484230 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:16:37.484281 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:37.484262 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:16:38.041212 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:38.041174 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" event={"ID":"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2","Type":"ContainerStarted","Data":"f702341b0591d0d69cbf6d69f9b380744a96c16925e0f0001a47ae5df0401383"} Jul 23 13:16:38.041608 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:38.041230 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" event={"ID":"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2","Type":"ContainerStarted","Data":"187a692ba49644f46fa33cc9855f0274125a858004db834d22b2873a125dfc69"} Jul 23 13:16:38.066387 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:38.066326 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" podStartSLOduration=2.066311971 podStartE2EDuration="2.066311971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:16:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:16:38.06421667 +0000 UTC m=+535.303492646" watchObservedRunningTime="2026-07-23 13:16:38.066311971 +0000 UTC m=+535.305587946" Jul 23 13:16:38.334419 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:38.334393 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:38.339733 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:38.339712 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:39.046498 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:39.046468 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:39.047324 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:39.047307 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:16:41.251623 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:41.251584 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:16:41.254174 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:41.254159 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-v4xtb" Jul 23 13:16:41.256540 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:41.256517 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5jhxm\"" Jul 23 13:16:41.256698 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:41.256661 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 23 13:16:41.263168 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:41.263145 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:16:41.289718 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:41.289693 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:16:41.380955 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:41.380921 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/50997322-eb3b-4098-ad0a-f9856d8b2c64-config-file\") pod \"limitador-limitador-54d6d988d5-v4xtb\" (UID: \"50997322-eb3b-4098-ad0a-f9856d8b2c64\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-v4xtb" Jul 23 13:16:41.381119 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:41.380983 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvl99\" (UniqueName: \"kubernetes.io/projected/50997322-eb3b-4098-ad0a-f9856d8b2c64-kube-api-access-lvl99\") pod \"limitador-limitador-54d6d988d5-v4xtb\" (UID: \"50997322-eb3b-4098-ad0a-f9856d8b2c64\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-v4xtb" Jul 23 13:16:41.481822 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:41.481788 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/50997322-eb3b-4098-ad0a-f9856d8b2c64-config-file\") pod \"limitador-limitador-54d6d988d5-v4xtb\" (UID: \"50997322-eb3b-4098-ad0a-f9856d8b2c64\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-v4xtb" Jul 23 13:16:41.482011 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:41.481843 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvl99\" (UniqueName: \"kubernetes.io/projected/50997322-eb3b-4098-ad0a-f9856d8b2c64-kube-api-access-lvl99\") pod \"limitador-limitador-54d6d988d5-v4xtb\" (UID: \"50997322-eb3b-4098-ad0a-f9856d8b2c64\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-v4xtb" Jul 23 13:16:41.482429 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:41.482408 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/50997322-eb3b-4098-ad0a-f9856d8b2c64-config-file\") pod \"limitador-limitador-54d6d988d5-v4xtb\" (UID: \"50997322-eb3b-4098-ad0a-f9856d8b2c64\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-v4xtb" Jul 23 13:16:41.490691 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:41.490665 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvl99\" (UniqueName: \"kubernetes.io/projected/50997322-eb3b-4098-ad0a-f9856d8b2c64-kube-api-access-lvl99\") pod \"limitador-limitador-54d6d988d5-v4xtb\" (UID: \"50997322-eb3b-4098-ad0a-f9856d8b2c64\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-v4xtb" Jul 23 13:16:41.565462 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:41.565429 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-v4xtb" Jul 23 13:16:41.703161 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:41.703138 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:16:41.705113 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:16:41.705076 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50997322_eb3b_4098_ad0a_f9856d8b2c64.slice/crio-449b6d3ae76842640aab1f2df1621f0c7294bf95455f45e883b0eda9190da11f WatchSource:0}: Error finding container 449b6d3ae76842640aab1f2df1621f0c7294bf95455f45e883b0eda9190da11f: Status 404 returned error can't find the container with id 449b6d3ae76842640aab1f2df1621f0c7294bf95455f45e883b0eda9190da11f Jul 23 13:16:42.062338 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:42.062301 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-v4xtb" event={"ID":"50997322-eb3b-4098-ad0a-f9856d8b2c64","Type":"ContainerStarted","Data":"449b6d3ae76842640aab1f2df1621f0c7294bf95455f45e883b0eda9190da11f"} Jul 23 13:16:47.092773 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:47.092740 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-v4xtb" event={"ID":"50997322-eb3b-4098-ad0a-f9856d8b2c64","Type":"ContainerStarted","Data":"21e1eb55523dbd58c1fe4cbb42b588d19a1181f26ff9a4438f6903e44c0c10c6"} Jul 23 13:16:47.093140 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:47.092801 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-v4xtb" Jul 23 13:16:47.124511 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:47.124463 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-v4xtb" podStartSLOduration=1.709292478 podStartE2EDuration="6.124447674s" totalImagesPullingTime="4.415155196s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:16:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:16:47.12420768 +0000 UTC m=+544.363483661" watchObservedRunningTime="2026-07-23 13:16:47.124447674 +0000 UTC m=+544.363723650" Jul 23 13:16:58.099526 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:16:58.099496 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-v4xtb" Jul 23 13:17:43.207054 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:17:43.207028 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6m8j_a0a5f6bc-bdb7-46ae-8cb9-79edd5179817/ovn-acl-logging/0.log" Jul 23 13:17:43.207584 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:17:43.207104 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6m8j_a0a5f6bc-bdb7-46ae-8cb9-79edd5179817/ovn-acl-logging/0.log" Jul 23 13:17:56.931478 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:17:56.931445 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:18:04.509044 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:04.508951 2531 generic.go:356] "Generic (PLEG): container finished" podID="ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2" containerID="f702341b0591d0d69cbf6d69f9b380744a96c16925e0f0001a47ae5df0401383" exitCode=0 Jul 23 13:18:04.509044 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:04.509010 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" event={"ID":"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2","Type":"ContainerDied","Data":"f702341b0591d0d69cbf6d69f9b380744a96c16925e0f0001a47ae5df0401383"} Jul 23 13:18:04.509510 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:04.509415 2531 scope.go:117] "RemoveContainer" containerID="f702341b0591d0d69cbf6d69f9b380744a96c16925e0f0001a47ae5df0401383" Jul 23 13:18:04.510499 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:04.510470 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:18:04.510589 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:04.510534 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:18:04.510589 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:04.510569 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:18:05.516868 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:05.516834 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" event={"ID":"ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2","Type":"ContainerStarted","Data":"1cefba2a4511d8970403a6d93bd2371c5e92cbf6de222d6ff6234877c7c34329"} Jul 23 13:18:06.334083 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:06.334042 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:18:06.335234 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:06.335198 2531 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-lxllp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 23 13:18:06.335398 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:06.335261 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" podUID="ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 23 13:18:07.334299 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:07.334257 2531 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-lxllp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 23 13:18:07.334819 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:07.334320 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" podUID="ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 23 13:18:08.334950 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:08.334907 2531 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-lxllp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 23 13:18:08.335455 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:08.334986 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" podUID="ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 23 13:18:09.047117 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:09.047076 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:18:09.334177 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:09.334148 2531 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-lxllp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 23 13:18:09.334404 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:09.334206 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" podUID="ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 23 13:18:10.337761 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:10.337730 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:18:10.338733 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:10.338713 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-lxllp" Jul 23 13:18:23.622033 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:23.621998 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:18:26.656909 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.656876 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5"] Jul 23 13:18:26.659885 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.659865 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" Jul 23 13:18:26.663070 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.663047 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 23 13:18:26.663194 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.663108 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 23 13:18:26.663194 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.663176 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 23 13:18:26.663309 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.663050 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7r8qz\"" Jul 23 13:18:26.671539 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.671514 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5"] Jul 23 13:18:26.727856 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.727817 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93ec1d6f-d14a-4df2-87eb-8bc69564a434-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w9fw5\" (UID: \"93ec1d6f-d14a-4df2-87eb-8bc69564a434\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" Jul 23 13:18:26.727856 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.727856 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93ec1d6f-d14a-4df2-87eb-8bc69564a434-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w9fw5\" (UID: \"93ec1d6f-d14a-4df2-87eb-8bc69564a434\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" Jul 23 13:18:26.728071 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.727978 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93ec1d6f-d14a-4df2-87eb-8bc69564a434-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w9fw5\" (UID: \"93ec1d6f-d14a-4df2-87eb-8bc69564a434\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" Jul 23 13:18:26.728071 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.728028 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93ec1d6f-d14a-4df2-87eb-8bc69564a434-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w9fw5\" (UID: \"93ec1d6f-d14a-4df2-87eb-8bc69564a434\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" Jul 23 13:18:26.728071 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.728061 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93ec1d6f-d14a-4df2-87eb-8bc69564a434-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w9fw5\" (UID: \"93ec1d6f-d14a-4df2-87eb-8bc69564a434\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" Jul 23 13:18:26.728187 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.728118 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb7zw\" (UniqueName: \"kubernetes.io/projected/93ec1d6f-d14a-4df2-87eb-8bc69564a434-kube-api-access-mb7zw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w9fw5\" (UID: \"93ec1d6f-d14a-4df2-87eb-8bc69564a434\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" Jul 23 13:18:26.829527 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.829489 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93ec1d6f-d14a-4df2-87eb-8bc69564a434-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w9fw5\" (UID: \"93ec1d6f-d14a-4df2-87eb-8bc69564a434\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" Jul 23 13:18:26.829527 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.829528 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93ec1d6f-d14a-4df2-87eb-8bc69564a434-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w9fw5\" (UID: \"93ec1d6f-d14a-4df2-87eb-8bc69564a434\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" Jul 23 13:18:26.829767 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.829564 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93ec1d6f-d14a-4df2-87eb-8bc69564a434-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w9fw5\" (UID: \"93ec1d6f-d14a-4df2-87eb-8bc69564a434\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" Jul 23 13:18:26.829767 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.829680 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93ec1d6f-d14a-4df2-87eb-8bc69564a434-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w9fw5\" (UID: \"93ec1d6f-d14a-4df2-87eb-8bc69564a434\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" Jul 23 13:18:26.829767 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.829714 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93ec1d6f-d14a-4df2-87eb-8bc69564a434-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w9fw5\" (UID: \"93ec1d6f-d14a-4df2-87eb-8bc69564a434\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" Jul 23 13:18:26.829977 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.829774 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb7zw\" (UniqueName: \"kubernetes.io/projected/93ec1d6f-d14a-4df2-87eb-8bc69564a434-kube-api-access-mb7zw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w9fw5\" (UID: \"93ec1d6f-d14a-4df2-87eb-8bc69564a434\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" Jul 23 13:18:26.829977 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.829956 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93ec1d6f-d14a-4df2-87eb-8bc69564a434-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w9fw5\" (UID: \"93ec1d6f-d14a-4df2-87eb-8bc69564a434\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" Jul 23 13:18:26.830159 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.830134 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93ec1d6f-d14a-4df2-87eb-8bc69564a434-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w9fw5\" (UID: \"93ec1d6f-d14a-4df2-87eb-8bc69564a434\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" Jul 23 13:18:26.830230 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.830182 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93ec1d6f-d14a-4df2-87eb-8bc69564a434-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w9fw5\" (UID: \"93ec1d6f-d14a-4df2-87eb-8bc69564a434\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" Jul 23 13:18:26.832111 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.832084 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93ec1d6f-d14a-4df2-87eb-8bc69564a434-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w9fw5\" (UID: \"93ec1d6f-d14a-4df2-87eb-8bc69564a434\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" Jul 23 13:18:26.832437 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.832404 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93ec1d6f-d14a-4df2-87eb-8bc69564a434-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w9fw5\" (UID: \"93ec1d6f-d14a-4df2-87eb-8bc69564a434\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" Jul 23 13:18:26.838553 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.838527 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb7zw\" (UniqueName: \"kubernetes.io/projected/93ec1d6f-d14a-4df2-87eb-8bc69564a434-kube-api-access-mb7zw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-w9fw5\" (UID: \"93ec1d6f-d14a-4df2-87eb-8bc69564a434\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" Jul 23 13:18:26.971031 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:26.970934 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" Jul 23 13:18:27.109618 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:27.109470 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5"] Jul 23 13:18:27.112567 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:27.112528 2531 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 13:18:27.486139 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:27.486102 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:18:27.641327 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:27.641288 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" event={"ID":"93ec1d6f-d14a-4df2-87eb-8bc69564a434","Type":"ContainerStarted","Data":"0a670c19851a7ca6d91d01c250a048c49bbb57ae47ade27a4acbe659dca425f3"} Jul 23 13:18:28.648426 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:28.648323 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" event={"ID":"93ec1d6f-d14a-4df2-87eb-8bc69564a434","Type":"ContainerStarted","Data":"a3fa14d1a699e123933034e930a908b9f5ef5f045e596470d864932810d6264a"} Jul 23 13:18:28.667417 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:28.667343 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" podStartSLOduration=1.526319915 podStartE2EDuration="2.667326996s" totalImagesPullingTime="1.141007081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:18:28.664842242 +0000 UTC m=+645.904118221" watchObservedRunningTime="2026-07-23 13:18:28.667326996 +0000 UTC m=+645.906602950" Jul 23 13:18:36.971631 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:36.971592 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" Jul 23 13:18:36.972086 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:36.971657 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" Jul 23 13:18:36.984260 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:36.984235 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" Jul 23 13:18:37.706349 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:37.706323 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-w9fw5" Jul 23 13:18:45.391511 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:18:45.391474 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:19:07.590572 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:19:07.590537 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:19:20.927818 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:19:20.927784 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:19:42.002934 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:19:42.002847 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:20:56.894052 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:20:56.894017 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:21:06.290286 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:21:06.290202 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:21:17.401715 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:21:17.401679 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:22:43.250896 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:22:43.250824 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6m8j_a0a5f6bc-bdb7-46ae-8cb9-79edd5179817/ovn-acl-logging/0.log" Jul 23 13:22:43.252767 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:22:43.252742 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6m8j_a0a5f6bc-bdb7-46ae-8cb9-79edd5179817/ovn-acl-logging/0.log" Jul 23 13:22:58.499241 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:22:58.499202 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:23:09.996415 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:23:09.996382 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:23:45.401508 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:23:45.401469 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:23:54.789457 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:23:54.789424 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:24:04.593447 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:24:04.593348 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:24:10.483615 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:24:10.483580 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:24:21.856742 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:24:21.856706 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:24:33.397315 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:24:33.397280 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:24:44.285996 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:24:44.285959 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:24:54.187398 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:24:54.187341 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:25:16.885716 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:25:16.885681 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:25:26.287993 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:25:26.287955 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:25:49.788126 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:25:49.788048 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:26:01.593729 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:26:01.593691 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:26:18.189944 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:26:18.189910 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:26:27.391153 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:26:27.391118 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:26:37.384007 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:26:37.383965 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:26:46.798217 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:26:46.798178 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:26:56.085558 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:26:56.085521 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:27:43.287095 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:27:43.287069 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6m8j_a0a5f6bc-bdb7-46ae-8cb9-79edd5179817/ovn-acl-logging/0.log" Jul 23 13:27:43.289755 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:27:43.289734 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6m8j_a0a5f6bc-bdb7-46ae-8cb9-79edd5179817/ovn-acl-logging/0.log" Jul 23 13:28:46.203384 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:28:46.203281 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:28:56.697578 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:28:56.697540 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:29:07.793969 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:29:07.793939 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:29:18.695103 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:29:18.695069 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:29:30.291688 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:29:30.291653 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:29:39.896176 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:29:39.896136 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:29:50.888978 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:29:50.888943 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:30:00.153520 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:30:00.153485 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29746890-hccpj"] Jul 23 13:30:00.156987 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:30:00.156971 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746890-hccpj" Jul 23 13:30:00.159571 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:30:00.159549 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vp6nn\"" Jul 23 13:30:00.166598 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:30:00.166572 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29746890-hccpj"] Jul 23 13:30:00.168316 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:30:00.168291 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746890-hccpj" Jul 23 13:30:00.308089 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:30:00.308063 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29746890-hccpj"] Jul 23 13:30:00.309464 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:30:00.309440 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb8f15f4_0f02_447e_94f9_0c8c709b59e7.slice/crio-6859eea16576bc08a4d07b90e4889f69e1a1a335b31a73d7577a758006dc7057 WatchSource:0}: Error finding container 6859eea16576bc08a4d07b90e4889f69e1a1a335b31a73d7577a758006dc7057: Status 404 returned error can't find the container with id 6859eea16576bc08a4d07b90e4889f69e1a1a335b31a73d7577a758006dc7057 Jul 23 13:30:00.311242 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:30:00.311224 2531 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 13:30:00.361297 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:30:00.361262 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746890-hccpj" event={"ID":"fb8f15f4-0f02-447e-94f9-0c8c709b59e7","Type":"ContainerStarted","Data":"6859eea16576bc08a4d07b90e4889f69e1a1a335b31a73d7577a758006dc7057"} Jul 23 13:30:00.696433 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:30:00.696398 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:30:03.383226 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:30:03.383192 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746890-hccpj" event={"ID":"fb8f15f4-0f02-447e-94f9-0c8c709b59e7","Type":"ContainerStarted","Data":"d5b3fd239b3372704308a147ce6634beab6941645cee8ba7b5b5cd6e8bf313a7"} Jul 23 13:30:03.402163 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:30:03.402107 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746890-hccpj" podStartSLOduration=1.181795407 podStartE2EDuration="3.402091713s" totalImagesPullingTime="2.220296306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:30:03.40029774 +0000 UTC m=+1340.639573716" watchObservedRunningTime="2026-07-23 13:30:03.402091713 +0000 UTC m=+1340.641367688" Jul 23 13:30:04.391193 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:30:04.391158 2531 generic.go:356] "Generic (PLEG): container finished" podID="fb8f15f4-0f02-447e-94f9-0c8c709b59e7" containerID="d5b3fd239b3372704308a147ce6634beab6941645cee8ba7b5b5cd6e8bf313a7" exitCode=0 Jul 23 13:30:04.391637 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:30:04.391245 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746890-hccpj" event={"ID":"fb8f15f4-0f02-447e-94f9-0c8c709b59e7","Type":"ContainerDied","Data":"d5b3fd239b3372704308a147ce6634beab6941645cee8ba7b5b5cd6e8bf313a7"} Jul 23 13:30:05.531716 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:30:05.531693 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746890-hccpj" Jul 23 13:30:06.403002 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:30:06.402974 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746890-hccpj" Jul 23 13:30:06.403177 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:30:06.402964 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746890-hccpj" event={"ID":"fb8f15f4-0f02-447e-94f9-0c8c709b59e7","Type":"ContainerDied","Data":"6859eea16576bc08a4d07b90e4889f69e1a1a335b31a73d7577a758006dc7057"} Jul 23 13:30:06.403177 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:30:06.403096 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6859eea16576bc08a4d07b90e4889f69e1a1a335b31a73d7577a758006dc7057" Jul 23 13:30:12.394362 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:30:12.394317 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:30:21.089344 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:30:21.089307 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:30:31.424215 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:30:31.424180 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:30:53.786427 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:30:53.786392 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:31:03.389633 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:31:03.389598 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:31:15.694868 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:31:15.694832 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:31:30.889299 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:31:30.889267 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:31:52.680221 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:31:52.680137 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:32:06.890781 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:32:06.890731 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:32:13.413569 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:32:13.413531 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:32:29.897795 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:32:29.897756 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:32:41.181584 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:32:41.181550 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:32:43.324223 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:32:43.324194 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6m8j_a0a5f6bc-bdb7-46ae-8cb9-79edd5179817/ovn-acl-logging/0.log" Jul 23 13:32:43.327598 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:32:43.327577 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6m8j_a0a5f6bc-bdb7-46ae-8cb9-79edd5179817/ovn-acl-logging/0.log" Jul 23 13:32:50.086054 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:32:50.086018 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:33:00.288817 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:33:00.288785 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:33:09.496770 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:33:09.496689 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:33:19.494157 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:33:19.494124 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:33:32.987222 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:33:32.987185 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:33:41.991395 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:33:41.991346 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:33:51.901861 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:33:51.901823 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:34:04.792348 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:34:04.792315 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:34:17.698719 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:34:17.698682 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:34:25.692887 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:34:25.692852 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:34:34.481465 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:34:34.481387 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:34:51.106004 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:34:51.105967 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:35:01.896442 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:01.896408 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:35:15.195348 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:15.195307 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:35:24.894874 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:24.894836 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:35:34.765478 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:34.765436 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:35:35.156593 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.156558 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh"] Jul 23 13:35:35.156976 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.156963 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb8f15f4-0f02-447e-94f9-0c8c709b59e7" containerName="cleanup" Jul 23 13:35:35.157023 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.156978 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb8f15f4-0f02-447e-94f9-0c8c709b59e7" containerName="cleanup" Jul 23 13:35:35.157064 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.157054 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb8f15f4-0f02-447e-94f9-0c8c709b59e7" containerName="cleanup" Jul 23 13:35:35.159274 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.159258 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.161761 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.161741 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e153fac9-openshift-default-dockercfg-9hb5s\"" Jul 23 13:35:35.184993 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.184923 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh"] Jul 23 13:35:35.191931 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.191906 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-credential-socket\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.192086 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.191937 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-workload-socket\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.192086 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.191956 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bff74b6d-d450-4675-afbd-5e33a4192336-istio-podinfo\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.192086 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.192038 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-workload-certs\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.192086 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.192070 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9tdk\" (UniqueName: \"kubernetes.io/projected/bff74b6d-d450-4675-afbd-5e33a4192336-kube-api-access-f9tdk\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.192330 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.192105 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-istio-envoy\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.192330 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.192135 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bff74b6d-d450-4675-afbd-5e33a4192336-istiod-ca-cert\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.192330 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.192157 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bff74b6d-d450-4675-afbd-5e33a4192336-istio-token\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.192330 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.192187 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-istio-data\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.292847 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.292813 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-credential-socket\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.292847 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.292849 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-workload-socket\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.293098 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.292867 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bff74b6d-d450-4675-afbd-5e33a4192336-istio-podinfo\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.293098 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.293001 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-workload-certs\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.293098 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.293041 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9tdk\" (UniqueName: \"kubernetes.io/projected/bff74b6d-d450-4675-afbd-5e33a4192336-kube-api-access-f9tdk\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.293098 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.293085 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-istio-envoy\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.293317 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.293104 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bff74b6d-d450-4675-afbd-5e33a4192336-istiod-ca-cert\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.293317 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.293125 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bff74b6d-d450-4675-afbd-5e33a4192336-istio-token\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.293317 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.293166 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-istio-data\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.293317 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.293219 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-workload-socket\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.293317 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.293294 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-credential-socket\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.293619 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.293330 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-workload-certs\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.293619 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.293568 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-istio-data\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.293935 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.293912 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bff74b6d-d450-4675-afbd-5e33a4192336-istiod-ca-cert\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.295483 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.295460 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-istio-envoy\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.295596 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.295558 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bff74b6d-d450-4675-afbd-5e33a4192336-istio-podinfo\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.302641 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.302616 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bff74b6d-d450-4675-afbd-5e33a4192336-istio-token\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.302747 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.302680 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9tdk\" (UniqueName: \"kubernetes.io/projected/bff74b6d-d450-4675-afbd-5e33a4192336-kube-api-access-f9tdk\") pod \"e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.469581 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.469476 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:35.616509 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.616468 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh"] Jul 23 13:35:35.617847 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:35:35.617813 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbff74b6d_d450_4675_afbd_5e33a4192336.slice/crio-2caa5bf6956b1b0973ffdc0f9a97f5fca9da53f66a8a9714c11d8219527431e8 WatchSource:0}: Error finding container 2caa5bf6956b1b0973ffdc0f9a97f5fca9da53f66a8a9714c11d8219527431e8: Status 404 returned error can't find the container with id 2caa5bf6956b1b0973ffdc0f9a97f5fca9da53f66a8a9714c11d8219527431e8 Jul 23 13:35:35.621749 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.620401 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:35:35.621749 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.620558 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:35:35.621749 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:35.620644 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:35:36.184507 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:36.184463 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" event={"ID":"bff74b6d-d450-4675-afbd-5e33a4192336","Type":"ContainerStarted","Data":"51e602802eba277b12d80485f14a0149b117ac06e57b7102e107f024a76a1d41"} Jul 23 13:35:36.184955 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:36.184546 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" event={"ID":"bff74b6d-d450-4675-afbd-5e33a4192336","Type":"ContainerStarted","Data":"2caa5bf6956b1b0973ffdc0f9a97f5fca9da53f66a8a9714c11d8219527431e8"} Jul 23 13:35:36.211093 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:36.211044 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" podStartSLOduration=1.211030788 podStartE2EDuration="1.211030788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:35:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:35:36.208387381 +0000 UTC m=+1673.447663351" watchObservedRunningTime="2026-07-23 13:35:36.211030788 +0000 UTC m=+1673.450306764" Jul 23 13:35:36.469960 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:36.469861 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:36.474720 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:36.474699 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:37.191410 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:37.191384 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:37.192308 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:37.192290 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:44.408532 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:44.408494 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:35:52.206721 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:52.206683 2531 patch_prober.go:28] interesting pod/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": read tcp 10.134.0.2:48588->10.134.0.46:15021: read: connection reset by peer" start-of-body= Jul 23 13:35:52.207141 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:52.206763 2531 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" podUID="bff74b6d-d450-4675-afbd-5e33a4192336" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": read tcp 10.134.0.2:48588->10.134.0.46:15021: read: connection reset by peer" Jul 23 13:35:52.283557 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:52.283529 2531 generic.go:356] "Generic (PLEG): container finished" podID="bff74b6d-d450-4675-afbd-5e33a4192336" containerID="51e602802eba277b12d80485f14a0149b117ac06e57b7102e107f024a76a1d41" exitCode=0 Jul 23 13:35:52.283717 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:52.283572 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" event={"ID":"bff74b6d-d450-4675-afbd-5e33a4192336","Type":"ContainerDied","Data":"51e602802eba277b12d80485f14a0149b117ac06e57b7102e107f024a76a1d41"} Jul 23 13:35:52.283984 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:52.283969 2531 scope.go:117] "RemoveContainer" containerID="51e602802eba277b12d80485f14a0149b117ac06e57b7102e107f024a76a1d41" Jul 23 13:35:52.285075 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:52.285039 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:35:52.285191 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:52.285131 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:35:52.285191 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:52.285180 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:35:53.290543 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:53.290508 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" event={"ID":"bff74b6d-d450-4675-afbd-5e33a4192336","Type":"ContainerStarted","Data":"237571844a2bac7f02d9122f64f07e579b1e078a7f7cf9411a12d0b1ff0177b0"} Jul 23 13:35:53.470303 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:53.470263 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:53.470563 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:53.470540 2531 patch_prober.go:28] interesting pod/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 23 13:35:53.470661 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:53.470594 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" podUID="bff74b6d-d450-4675-afbd-5e33a4192336" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 23 13:35:54.470777 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:54.470740 2531 patch_prober.go:28] interesting pod/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 23 13:35:54.471181 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:54.470807 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" podUID="bff74b6d-d450-4675-afbd-5e33a4192336" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 23 13:35:55.470313 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:55.470280 2531 patch_prober.go:28] interesting pod/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 23 13:35:55.470518 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:55.470338 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" podUID="bff74b6d-d450-4675-afbd-5e33a4192336" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 23 13:35:56.470835 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:56.470793 2531 patch_prober.go:28] interesting pod/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 23 13:35:56.471228 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:56.470856 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" podUID="bff74b6d-d450-4675-afbd-5e33a4192336" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 23 13:35:57.473864 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:57.473832 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:57.474341 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:57.474125 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:35:57.474849 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:35:57.474832 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:36:03.319439 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:03.319401 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e153fac9-r47fj"] Jul 23 13:36:03.322065 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:03.322048 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e153fac9-r47fj" Jul 23 13:36:03.328234 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:03.328204 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 13:36:03.328464 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:03.328445 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vp6nn\"" Jul 23 13:36:03.349894 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:03.349862 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e153fac9-r47fj"] Jul 23 13:36:03.456803 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:03.456727 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c781e598-a698-45f2-9983-f879c0516b13-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e153fac9-r47fj\" (UID: \"c781e598-a698-45f2-9983-f879c0516b13\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e153fac9-r47fj" Jul 23 13:36:03.557382 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:03.557317 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c781e598-a698-45f2-9983-f879c0516b13-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e153fac9-r47fj\" (UID: \"c781e598-a698-45f2-9983-f879c0516b13\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e153fac9-r47fj" Jul 23 13:36:03.558002 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:03.557983 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c781e598-a698-45f2-9983-f879c0516b13-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e153fac9-r47fj\" (UID: \"c781e598-a698-45f2-9983-f879c0516b13\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e153fac9-r47fj" Jul 23 13:36:03.632841 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:03.632753 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e153fac9-r47fj" Jul 23 13:36:03.804685 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:03.804653 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e153fac9-r47fj"] Jul 23 13:36:03.809763 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:36:03.809729 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc781e598_a698_45f2_9983_f879c0516b13.slice/crio-d7293d0ff8ad92b72d96df15e5d7d6c219e43c55bcf5f8f998d2bdb3fff6e87b WatchSource:0}: Error finding container d7293d0ff8ad92b72d96df15e5d7d6c219e43c55bcf5f8f998d2bdb3fff6e87b: Status 404 returned error can't find the container with id d7293d0ff8ad92b72d96df15e5d7d6c219e43c55bcf5f8f998d2bdb3fff6e87b Jul 23 13:36:04.355864 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:04.355774 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e153fac9-r47fj" event={"ID":"c781e598-a698-45f2-9983-f879c0516b13","Type":"ContainerStarted","Data":"29434c086890a29f3c42fc9ca5df2a8bfa5b44d964e24b5a1da50e1eba29f150"} Jul 23 13:36:04.355864 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:04.355829 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e153fac9-r47fj" event={"ID":"c781e598-a698-45f2-9983-f879c0516b13","Type":"ContainerStarted","Data":"d7293d0ff8ad92b72d96df15e5d7d6c219e43c55bcf5f8f998d2bdb3fff6e87b"} Jul 23 13:36:04.402046 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:04.401993 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e153fac9-r47fj" podStartSLOduration=1.401978752 podStartE2EDuration="1.401978752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:36:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:36:04.400633415 +0000 UTC m=+1701.639909392" watchObservedRunningTime="2026-07-23 13:36:04.401978752 +0000 UTC m=+1701.641254727" Jul 23 13:36:05.361983 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:05.361951 2531 generic.go:356] "Generic (PLEG): container finished" podID="c781e598-a698-45f2-9983-f879c0516b13" containerID="29434c086890a29f3c42fc9ca5df2a8bfa5b44d964e24b5a1da50e1eba29f150" exitCode=0 Jul 23 13:36:05.362462 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:05.362040 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e153fac9-r47fj" event={"ID":"c781e598-a698-45f2-9983-f879c0516b13","Type":"ContainerDied","Data":"29434c086890a29f3c42fc9ca5df2a8bfa5b44d964e24b5a1da50e1eba29f150"} Jul 23 13:36:06.500952 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:06.500928 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e153fac9-r47fj" Jul 23 13:36:06.688814 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:06.688724 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c781e598-a698-45f2-9983-f879c0516b13-maas-api-service-ca\") pod \"c781e598-a698-45f2-9983-f879c0516b13\" (UID: \"c781e598-a698-45f2-9983-f879c0516b13\") " Jul 23 13:36:06.689091 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:06.689067 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c781e598-a698-45f2-9983-f879c0516b13-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c781e598-a698-45f2-9983-f879c0516b13" (UID: "c781e598-a698-45f2-9983-f879c0516b13"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:36:06.789694 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:06.789661 2531 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c781e598-a698-45f2-9983-f879c0516b13-maas-api-service-ca\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:36:07.374511 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:07.374480 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e153fac9-r47fj" Jul 23 13:36:07.374679 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:07.374481 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e153fac9-r47fj" event={"ID":"c781e598-a698-45f2-9983-f879c0516b13","Type":"ContainerDied","Data":"d7293d0ff8ad92b72d96df15e5d7d6c219e43c55bcf5f8f998d2bdb3fff6e87b"} Jul 23 13:36:07.374679 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:07.374613 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d7293d0ff8ad92b72d96df15e5d7d6c219e43c55bcf5f8f998d2bdb3fff6e87b" Jul 23 13:36:16.927802 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:16.927763 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:36:18.443115 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:18.443082 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e153fac9-r47fj"] Jul 23 13:36:18.456469 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:18.456439 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e153fac9-r47fj"] Jul 23 13:36:18.634867 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:18.634832 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh"] Jul 23 13:36:18.635114 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:18.635090 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" podUID="bff74b6d-d450-4675-afbd-5e33a4192336" containerName="istio-proxy" containerID="cri-o://237571844a2bac7f02d9122f64f07e579b1e078a7f7cf9411a12d0b1ff0177b0" gracePeriod=30 Jul 23 13:36:19.332144 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:19.332115 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c781e598-a698-45f2-9983-f879c0516b13" path="/var/lib/kubelet/pods/c781e598-a698-45f2-9983-f879c0516b13/volumes" Jul 23 13:36:20.038658 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.038622 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp"] Jul 23 13:36:20.039071 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.039055 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c781e598-a698-45f2-9983-f879c0516b13" containerName="revoke-keys" Jul 23 13:36:20.039121 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.039073 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="c781e598-a698-45f2-9983-f879c0516b13" containerName="revoke-keys" Jul 23 13:36:20.039167 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.039157 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="c781e598-a698-45f2-9983-f879c0516b13" containerName="revoke-keys" Jul 23 13:36:20.042368 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.042336 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.057324 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.057294 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-6915d69d-gw-openshift-default-dockercfg-dgm8v\"" Jul 23 13:36:20.073942 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.073914 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp"] Jul 23 13:36:20.100495 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.100459 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-workload-certs\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.100495 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.100500 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29033909-c825-4062-a368-2e1c043e6a99-istiod-ca-cert\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.100702 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.100523 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-istio-data\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.100702 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.100624 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-istio-envoy\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.100702 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.100669 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr8lg\" (UniqueName: \"kubernetes.io/projected/29033909-c825-4062-a368-2e1c043e6a99-kube-api-access-dr8lg\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.100812 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.100724 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-workload-socket\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.100812 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.100798 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29033909-c825-4062-a368-2e1c043e6a99-istio-podinfo\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.100876 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.100828 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-credential-socket\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.100876 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.100858 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29033909-c825-4062-a368-2e1c043e6a99-istio-token\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.201978 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.201943 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29033909-c825-4062-a368-2e1c043e6a99-istio-podinfo\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.201978 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.201985 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-credential-socket\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.202220 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.202006 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29033909-c825-4062-a368-2e1c043e6a99-istio-token\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.202220 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.202066 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-workload-certs\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.202220 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.202081 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29033909-c825-4062-a368-2e1c043e6a99-istiod-ca-cert\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.202220 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.202101 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-istio-data\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.202482 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.202213 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-istio-envoy\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.202482 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.202289 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr8lg\" (UniqueName: \"kubernetes.io/projected/29033909-c825-4062-a368-2e1c043e6a99-kube-api-access-dr8lg\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.202482 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.202320 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-workload-socket\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.202636 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.202514 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-workload-certs\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.202636 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.202585 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-istio-data\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.202746 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.202649 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-credential-socket\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.202803 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.202753 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-workload-socket\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.202978 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.202957 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29033909-c825-4062-a368-2e1c043e6a99-istiod-ca-cert\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.204524 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.204479 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-istio-envoy\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.204783 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.204764 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29033909-c825-4062-a368-2e1c043e6a99-istio-podinfo\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.222013 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.221968 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29033909-c825-4062-a368-2e1c043e6a99-istio-token\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.222211 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.222189 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr8lg\" (UniqueName: \"kubernetes.io/projected/29033909-c825-4062-a368-2e1c043e6a99-kube-api-access-dr8lg\") pod \"e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.352099 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.352062 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:20.512454 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.512425 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp"] Jul 23 13:36:20.513550 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:36:20.513520 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29033909_c825_4062_a368_2e1c043e6a99.slice/crio-6110bb0ac57cf2afae76f5ec978fdcde84c3a25363e0ff0d800f9b59606c433c WatchSource:0}: Error finding container 6110bb0ac57cf2afae76f5ec978fdcde84c3a25363e0ff0d800f9b59606c433c: Status 404 returned error can't find the container with id 6110bb0ac57cf2afae76f5ec978fdcde84c3a25363e0ff0d800f9b59606c433c Jul 23 13:36:20.515532 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.515503 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:36:20.515583 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.515570 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:36:20.515622 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:20.515597 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:36:21.455948 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:21.455909 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" event={"ID":"29033909-c825-4062-a368-2e1c043e6a99","Type":"ContainerStarted","Data":"552a97eab696c632f976db5d96531a3af3ad58d3e96afb9530dcd35cfc22fc56"} Jul 23 13:36:21.456448 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:21.455962 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" event={"ID":"29033909-c825-4062-a368-2e1c043e6a99","Type":"ContainerStarted","Data":"6110bb0ac57cf2afae76f5ec978fdcde84c3a25363e0ff0d800f9b59606c433c"} Jul 23 13:36:21.482090 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:21.482042 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" podStartSLOduration=2.482025644 podStartE2EDuration="2.482025644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:36:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:36:21.478709758 +0000 UTC m=+1718.717985736" watchObservedRunningTime="2026-07-23 13:36:21.482025644 +0000 UTC m=+1718.721301620" Jul 23 13:36:22.352698 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:22.352667 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:22.357712 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:22.357687 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:22.462417 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:22.462391 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:22.463420 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:22.463402 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:23.887204 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:23.887179 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:36:23.946555 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:23.946477 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-workload-socket\") pod \"bff74b6d-d450-4675-afbd-5e33a4192336\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " Jul 23 13:36:23.946555 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:23.946514 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-workload-certs\") pod \"bff74b6d-d450-4675-afbd-5e33a4192336\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " Jul 23 13:36:23.946555 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:23.946536 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bff74b6d-d450-4675-afbd-5e33a4192336-istio-token\") pod \"bff74b6d-d450-4675-afbd-5e33a4192336\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " Jul 23 13:36:23.946779 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:23.946595 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-istio-envoy\") pod \"bff74b6d-d450-4675-afbd-5e33a4192336\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " Jul 23 13:36:23.946779 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:23.946661 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9tdk\" (UniqueName: \"kubernetes.io/projected/bff74b6d-d450-4675-afbd-5e33a4192336-kube-api-access-f9tdk\") pod \"bff74b6d-d450-4675-afbd-5e33a4192336\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " Jul 23 13:36:23.946779 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:23.946694 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-credential-socket\") pod \"bff74b6d-d450-4675-afbd-5e33a4192336\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " Jul 23 13:36:23.946779 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:23.946749 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bff74b6d-d450-4675-afbd-5e33a4192336-istio-podinfo\") pod \"bff74b6d-d450-4675-afbd-5e33a4192336\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " Jul 23 13:36:23.946983 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:23.946767 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bff74b6d-d450-4675-afbd-5e33a4192336" (UID: "bff74b6d-d450-4675-afbd-5e33a4192336"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:36:23.946983 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:23.946786 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bff74b6d-d450-4675-afbd-5e33a4192336-istiod-ca-cert\") pod \"bff74b6d-d450-4675-afbd-5e33a4192336\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " Jul 23 13:36:23.946983 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:23.946834 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-istio-data\") pod \"bff74b6d-d450-4675-afbd-5e33a4192336\" (UID: \"bff74b6d-d450-4675-afbd-5e33a4192336\") " Jul 23 13:36:23.946983 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:23.946855 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bff74b6d-d450-4675-afbd-5e33a4192336" (UID: "bff74b6d-d450-4675-afbd-5e33a4192336"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:36:23.947185 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:23.947055 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bff74b6d-d450-4675-afbd-5e33a4192336" (UID: "bff74b6d-d450-4675-afbd-5e33a4192336"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:36:23.947293 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:23.947258 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bff74b6d-d450-4675-afbd-5e33a4192336-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bff74b6d-d450-4675-afbd-5e33a4192336" (UID: "bff74b6d-d450-4675-afbd-5e33a4192336"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:36:23.947495 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:23.947476 2531 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-credential-socket\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:36:23.947655 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:23.947500 2531 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bff74b6d-d450-4675-afbd-5e33a4192336-istiod-ca-cert\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:36:23.947655 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:23.947516 2531 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-workload-socket\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:36:23.947655 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:23.947529 2531 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-workload-certs\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:36:23.947930 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:23.947897 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-istio-data" (OuterVolumeSpecName: "istio-data") pod "bff74b6d-d450-4675-afbd-5e33a4192336" (UID: "bff74b6d-d450-4675-afbd-5e33a4192336"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:36:23.949256 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:23.949219 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bff74b6d-d450-4675-afbd-5e33a4192336" (UID: "bff74b6d-d450-4675-afbd-5e33a4192336"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:36:23.949388 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:23.949312 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bff74b6d-d450-4675-afbd-5e33a4192336-kube-api-access-f9tdk" (OuterVolumeSpecName: "kube-api-access-f9tdk") pod "bff74b6d-d450-4675-afbd-5e33a4192336" (UID: "bff74b6d-d450-4675-afbd-5e33a4192336"). InnerVolumeSpecName "kube-api-access-f9tdk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:36:23.949555 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:23.949534 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bff74b6d-d450-4675-afbd-5e33a4192336-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bff74b6d-d450-4675-afbd-5e33a4192336" (UID: "bff74b6d-d450-4675-afbd-5e33a4192336"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 13:36:23.949639 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:23.949558 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bff74b6d-d450-4675-afbd-5e33a4192336-istio-token" (OuterVolumeSpecName: "istio-token") pod "bff74b6d-d450-4675-afbd-5e33a4192336" (UID: "bff74b6d-d450-4675-afbd-5e33a4192336"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:36:24.048819 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:24.048786 2531 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-istio-data\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:36:24.048819 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:24.048811 2531 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bff74b6d-d450-4675-afbd-5e33a4192336-istio-token\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:36:24.048819 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:24.048820 2531 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bff74b6d-d450-4675-afbd-5e33a4192336-istio-envoy\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:36:24.049047 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:24.048829 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9tdk\" (UniqueName: \"kubernetes.io/projected/bff74b6d-d450-4675-afbd-5e33a4192336-kube-api-access-f9tdk\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:36:24.049047 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:24.048838 2531 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bff74b6d-d450-4675-afbd-5e33a4192336-istio-podinfo\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:36:24.475203 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:24.475170 2531 generic.go:356] "Generic (PLEG): container finished" podID="bff74b6d-d450-4675-afbd-5e33a4192336" containerID="237571844a2bac7f02d9122f64f07e579b1e078a7f7cf9411a12d0b1ff0177b0" exitCode=0 Jul 23 13:36:24.475391 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:24.475274 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" Jul 23 13:36:24.475391 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:24.475306 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" event={"ID":"bff74b6d-d450-4675-afbd-5e33a4192336","Type":"ContainerDied","Data":"237571844a2bac7f02d9122f64f07e579b1e078a7f7cf9411a12d0b1ff0177b0"} Jul 23 13:36:24.475391 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:24.475378 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh" event={"ID":"bff74b6d-d450-4675-afbd-5e33a4192336","Type":"ContainerDied","Data":"2caa5bf6956b1b0973ffdc0f9a97f5fca9da53f66a8a9714c11d8219527431e8"} Jul 23 13:36:24.475573 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:24.475402 2531 scope.go:117] "RemoveContainer" containerID="237571844a2bac7f02d9122f64f07e579b1e078a7f7cf9411a12d0b1ff0177b0" Jul 23 13:36:24.485451 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:24.485431 2531 scope.go:117] "RemoveContainer" containerID="51e602802eba277b12d80485f14a0149b117ac06e57b7102e107f024a76a1d41" Jul 23 13:36:24.493898 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:24.493882 2531 scope.go:117] "RemoveContainer" containerID="237571844a2bac7f02d9122f64f07e579b1e078a7f7cf9411a12d0b1ff0177b0" Jul 23 13:36:24.494167 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:36:24.494146 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"237571844a2bac7f02d9122f64f07e579b1e078a7f7cf9411a12d0b1ff0177b0\": container with ID starting with 237571844a2bac7f02d9122f64f07e579b1e078a7f7cf9411a12d0b1ff0177b0 not found: ID does not exist" containerID="237571844a2bac7f02d9122f64f07e579b1e078a7f7cf9411a12d0b1ff0177b0" Jul 23 13:36:24.494249 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:24.494179 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"237571844a2bac7f02d9122f64f07e579b1e078a7f7cf9411a12d0b1ff0177b0"} err="failed to get container status \"237571844a2bac7f02d9122f64f07e579b1e078a7f7cf9411a12d0b1ff0177b0\": rpc error: code = NotFound desc = could not find container \"237571844a2bac7f02d9122f64f07e579b1e078a7f7cf9411a12d0b1ff0177b0\": container with ID starting with 237571844a2bac7f02d9122f64f07e579b1e078a7f7cf9411a12d0b1ff0177b0 not found: ID does not exist" Jul 23 13:36:24.494249 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:24.494202 2531 scope.go:117] "RemoveContainer" containerID="51e602802eba277b12d80485f14a0149b117ac06e57b7102e107f024a76a1d41" Jul 23 13:36:24.494468 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:36:24.494451 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51e602802eba277b12d80485f14a0149b117ac06e57b7102e107f024a76a1d41\": container with ID starting with 51e602802eba277b12d80485f14a0149b117ac06e57b7102e107f024a76a1d41 not found: ID does not exist" containerID="51e602802eba277b12d80485f14a0149b117ac06e57b7102e107f024a76a1d41" Jul 23 13:36:24.494524 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:24.494475 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51e602802eba277b12d80485f14a0149b117ac06e57b7102e107f024a76a1d41"} err="failed to get container status \"51e602802eba277b12d80485f14a0149b117ac06e57b7102e107f024a76a1d41\": rpc error: code = NotFound desc = could not find container \"51e602802eba277b12d80485f14a0149b117ac06e57b7102e107f024a76a1d41\": container with ID starting with 51e602802eba277b12d80485f14a0149b117ac06e57b7102e107f024a76a1d41 not found: ID does not exist" Jul 23 13:36:24.505791 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:24.505766 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh"] Jul 23 13:36:24.508520 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:24.508494 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e153fac9-openshift-default-7858ff54fc-2wkfh"] Jul 23 13:36:25.333007 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:25.332974 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bff74b6d-d450-4675-afbd-5e33a4192336" path="/var/lib/kubelet/pods/bff74b6d-d450-4675-afbd-5e33a4192336/volumes" Jul 23 13:36:27.495148 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:27.495115 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:36:35.540007 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:35.539917 2531 generic.go:356] "Generic (PLEG): container finished" podID="29033909-c825-4062-a368-2e1c043e6a99" containerID="552a97eab696c632f976db5d96531a3af3ad58d3e96afb9530dcd35cfc22fc56" exitCode=0 Jul 23 13:36:35.540007 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:35.539952 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" event={"ID":"29033909-c825-4062-a368-2e1c043e6a99","Type":"ContainerDied","Data":"552a97eab696c632f976db5d96531a3af3ad58d3e96afb9530dcd35cfc22fc56"} Jul 23 13:36:35.540470 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:35.540314 2531 scope.go:117] "RemoveContainer" containerID="552a97eab696c632f976db5d96531a3af3ad58d3e96afb9530dcd35cfc22fc56" Jul 23 13:36:35.541307 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:35.541282 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:36:35.541379 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:35.541344 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:36:35.541428 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:35.541391 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:36:36.554883 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:36.554844 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" event={"ID":"29033909-c825-4062-a368-2e1c043e6a99","Type":"ContainerStarted","Data":"d0e87770b73cbe9914a69a18725567baefd91620568cec923498c67209e3ba17"} Jul 23 13:36:37.352501 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:37.352459 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:37.353625 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:37.353590 2531 patch_prober.go:28] interesting pod/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 23 13:36:37.353756 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:37.353663 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" podUID="29033909-c825-4062-a368-2e1c043e6a99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 23 13:36:37.462577 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:37.462540 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:38.353058 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:38.353017 2531 patch_prober.go:28] interesting pod/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 23 13:36:38.353692 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:38.353659 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" podUID="29033909-c825-4062-a368-2e1c043e6a99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 23 13:36:39.352897 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:39.352457 2531 patch_prober.go:28] interesting pod/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 23 13:36:39.352897 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:39.352528 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" podUID="29033909-c825-4062-a368-2e1c043e6a99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 23 13:36:40.353216 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:40.353170 2531 patch_prober.go:28] interesting pod/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 23 13:36:40.353739 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:40.353240 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" podUID="29033909-c825-4062-a368-2e1c043e6a99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 23 13:36:41.356659 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:41.356631 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:41.357523 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:41.357506 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:36:48.192332 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:48.192298 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6915d69d-tk6tt"] Jul 23 13:36:48.192748 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:48.192733 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bff74b6d-d450-4675-afbd-5e33a4192336" containerName="istio-proxy" Jul 23 13:36:48.192796 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:48.192750 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="bff74b6d-d450-4675-afbd-5e33a4192336" containerName="istio-proxy" Jul 23 13:36:48.192796 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:48.192764 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bff74b6d-d450-4675-afbd-5e33a4192336" containerName="istio-proxy" Jul 23 13:36:48.192796 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:48.192770 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="bff74b6d-d450-4675-afbd-5e33a4192336" containerName="istio-proxy" Jul 23 13:36:48.192888 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:48.192846 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="bff74b6d-d450-4675-afbd-5e33a4192336" containerName="istio-proxy" Jul 23 13:36:48.192888 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:48.192855 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="bff74b6d-d450-4675-afbd-5e33a4192336" containerName="istio-proxy" Jul 23 13:36:48.197039 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:48.197023 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6915d69d-tk6tt" Jul 23 13:36:48.199731 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:48.199712 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vp6nn\"" Jul 23 13:36:48.199839 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:48.199740 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 13:36:48.206744 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:48.206718 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6915d69d-tk6tt"] Jul 23 13:36:48.383598 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:48.383562 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/855c02e4-cfa0-4b01-aa34-7a826ceea080-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6915d69d-tk6tt\" (UID: \"855c02e4-cfa0-4b01-aa34-7a826ceea080\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6915d69d-tk6tt" Jul 23 13:36:48.484425 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:48.484324 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/855c02e4-cfa0-4b01-aa34-7a826ceea080-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6915d69d-tk6tt\" (UID: \"855c02e4-cfa0-4b01-aa34-7a826ceea080\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6915d69d-tk6tt" Jul 23 13:36:48.484957 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:48.484935 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/855c02e4-cfa0-4b01-aa34-7a826ceea080-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6915d69d-tk6tt\" (UID: \"855c02e4-cfa0-4b01-aa34-7a826ceea080\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6915d69d-tk6tt" Jul 23 13:36:48.508335 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:48.508308 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6915d69d-tk6tt" Jul 23 13:36:48.656132 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:48.656104 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6915d69d-tk6tt"] Jul 23 13:36:48.657734 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:36:48.657705 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod855c02e4_cfa0_4b01_aa34_7a826ceea080.slice/crio-f9ba054ce660d93df9919e85e17e4fbef15c280ad7fdf133210bdb5f7f744a4e WatchSource:0}: Error finding container f9ba054ce660d93df9919e85e17e4fbef15c280ad7fdf133210bdb5f7f744a4e: Status 404 returned error can't find the container with id f9ba054ce660d93df9919e85e17e4fbef15c280ad7fdf133210bdb5f7f744a4e Jul 23 13:36:49.629597 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:49.629504 2531 generic.go:356] "Generic (PLEG): container finished" podID="855c02e4-cfa0-4b01-aa34-7a826ceea080" containerID="b06b54145e97eba8d3449a7c56d7413afbd7322974a3c3931d00e6a2aae9af0a" exitCode=0 Jul 23 13:36:49.629597 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:49.629585 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6915d69d-tk6tt" event={"ID":"855c02e4-cfa0-4b01-aa34-7a826ceea080","Type":"ContainerDied","Data":"b06b54145e97eba8d3449a7c56d7413afbd7322974a3c3931d00e6a2aae9af0a"} Jul 23 13:36:49.630026 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:49.629628 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6915d69d-tk6tt" event={"ID":"855c02e4-cfa0-4b01-aa34-7a826ceea080","Type":"ContainerStarted","Data":"f9ba054ce660d93df9919e85e17e4fbef15c280ad7fdf133210bdb5f7f744a4e"} Jul 23 13:36:50.773336 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:50.773312 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6915d69d-tk6tt" Jul 23 13:36:50.907288 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:50.907195 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/855c02e4-cfa0-4b01-aa34-7a826ceea080-maas-api-service-ca\") pod \"855c02e4-cfa0-4b01-aa34-7a826ceea080\" (UID: \"855c02e4-cfa0-4b01-aa34-7a826ceea080\") " Jul 23 13:36:50.907610 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:50.907590 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/855c02e4-cfa0-4b01-aa34-7a826ceea080-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "855c02e4-cfa0-4b01-aa34-7a826ceea080" (UID: "855c02e4-cfa0-4b01-aa34-7a826ceea080"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:36:51.008097 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:51.008061 2531 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/855c02e4-cfa0-4b01-aa34-7a826ceea080-maas-api-service-ca\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:36:51.641662 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:51.641573 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6915d69d-tk6tt" event={"ID":"855c02e4-cfa0-4b01-aa34-7a826ceea080","Type":"ContainerDied","Data":"f9ba054ce660d93df9919e85e17e4fbef15c280ad7fdf133210bdb5f7f744a4e"} Jul 23 13:36:51.641662 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:51.641612 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6915d69d-tk6tt" Jul 23 13:36:51.641662 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:36:51.641626 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f9ba054ce660d93df9919e85e17e4fbef15c280ad7fdf133210bdb5f7f744a4e" Jul 23 13:37:01.606513 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:01.606474 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:37:03.354743 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:03.354704 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6915d69d-tk6tt"] Jul 23 13:37:03.357249 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:03.357222 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6915d69d-tk6tt"] Jul 23 13:37:03.514865 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:03.514834 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp"] Jul 23 13:37:03.515071 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:03.515046 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" podUID="29033909-c825-4062-a368-2e1c043e6a99" containerName="istio-proxy" containerID="cri-o://d0e87770b73cbe9914a69a18725567baefd91620568cec923498c67209e3ba17" gracePeriod=30 Jul 23 13:37:04.467969 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.467939 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5"] Jul 23 13:37:04.468349 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.468335 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="855c02e4-cfa0-4b01-aa34-7a826ceea080" containerName="revoke-keys" Jul 23 13:37:04.468349 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.468346 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="855c02e4-cfa0-4b01-aa34-7a826ceea080" containerName="revoke-keys" Jul 23 13:37:04.468458 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.468420 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="855c02e4-cfa0-4b01-aa34-7a826ceea080" containerName="revoke-keys" Jul 23 13:37:04.471708 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.471690 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.477573 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.477553 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-94afaac2-openshift-default-dockercfg-z42vl\"" Jul 23 13:37:04.489331 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.489307 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5"] Jul 23 13:37:04.521290 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.521259 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr6dn\" (UniqueName: \"kubernetes.io/projected/00268503-c98a-4f85-817a-8289827f50bf-kube-api-access-cr6dn\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.521487 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.521304 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00268503-c98a-4f85-817a-8289827f50bf-istio-podinfo\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.521487 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.521327 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-istio-data\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.521487 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.521371 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00268503-c98a-4f85-817a-8289827f50bf-istiod-ca-cert\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.521487 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.521401 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-istio-envoy\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.521487 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.521423 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-workload-socket\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.521487 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.521459 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00268503-c98a-4f85-817a-8289827f50bf-istio-token\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.521690 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.521520 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-credential-socket\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.521690 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.521553 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-workload-certs\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.622177 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.622138 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-credential-socket\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.622177 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.622185 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-workload-certs\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.622435 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.622309 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr6dn\" (UniqueName: \"kubernetes.io/projected/00268503-c98a-4f85-817a-8289827f50bf-kube-api-access-cr6dn\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.622435 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.622406 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00268503-c98a-4f85-817a-8289827f50bf-istio-podinfo\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.622545 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.622438 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-istio-data\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.622545 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.622480 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00268503-c98a-4f85-817a-8289827f50bf-istiod-ca-cert\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.622545 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.622515 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-istio-envoy\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.622545 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.622541 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-workload-certs\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.622778 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.622548 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-workload-socket\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.622778 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.622592 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00268503-c98a-4f85-817a-8289827f50bf-istio-token\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.622778 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.622611 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-credential-socket\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.622947 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.622797 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-istio-data\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.622947 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.622842 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-workload-socket\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.623190 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.623171 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00268503-c98a-4f85-817a-8289827f50bf-istiod-ca-cert\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.624841 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.624819 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-istio-envoy\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.625067 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.625048 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00268503-c98a-4f85-817a-8289827f50bf-istio-podinfo\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.633240 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.633215 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr6dn\" (UniqueName: \"kubernetes.io/projected/00268503-c98a-4f85-817a-8289827f50bf-kube-api-access-cr6dn\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.634882 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.634858 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00268503-c98a-4f85-817a-8289827f50bf-istio-token\") pod \"e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.781574 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.781491 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:04.934545 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.934520 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5"] Jul 23 13:37:04.936116 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:37:04.936091 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00268503_c98a_4f85_817a_8289827f50bf.slice/crio-035d06c896715798ef42b1dc3ff61afc872c86a51d7dd02a94b38bbc2c6376c7 WatchSource:0}: Error finding container 035d06c896715798ef42b1dc3ff61afc872c86a51d7dd02a94b38bbc2c6376c7: Status 404 returned error can't find the container with id 035d06c896715798ef42b1dc3ff61afc872c86a51d7dd02a94b38bbc2c6376c7 Jul 23 13:37:04.938167 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.938133 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:37:04.938251 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.938197 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:37:04.938251 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:04.938225 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:37:05.333321 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:05.333285 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="855c02e4-cfa0-4b01-aa34-7a826ceea080" path="/var/lib/kubelet/pods/855c02e4-cfa0-4b01-aa34-7a826ceea080/volumes" Jul 23 13:37:05.726195 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:05.726091 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" event={"ID":"00268503-c98a-4f85-817a-8289827f50bf","Type":"ContainerStarted","Data":"31498a411f3a276595dcafc0dff62f0f08ac85d62dd00655aed442f8812ad6d6"} Jul 23 13:37:05.726195 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:05.726144 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" event={"ID":"00268503-c98a-4f85-817a-8289827f50bf","Type":"ContainerStarted","Data":"035d06c896715798ef42b1dc3ff61afc872c86a51d7dd02a94b38bbc2c6376c7"} Jul 23 13:37:05.764667 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:05.764614 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" podStartSLOduration=1.764597749 podStartE2EDuration="1.764597749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:37:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:37:05.761992265 +0000 UTC m=+1763.001268241" watchObservedRunningTime="2026-07-23 13:37:05.764597749 +0000 UTC m=+1763.003873725" Jul 23 13:37:05.781670 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:05.781642 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:05.786260 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:05.786238 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:06.731596 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:06.731565 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:06.732542 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:06.732523 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:08.746981 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.746953 2531 generic.go:356] "Generic (PLEG): container finished" podID="29033909-c825-4062-a368-2e1c043e6a99" containerID="d0e87770b73cbe9914a69a18725567baefd91620568cec923498c67209e3ba17" exitCode=0 Jul 23 13:37:08.747446 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.747024 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" event={"ID":"29033909-c825-4062-a368-2e1c043e6a99","Type":"ContainerDied","Data":"d0e87770b73cbe9914a69a18725567baefd91620568cec923498c67209e3ba17"} Jul 23 13:37:08.747446 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.747092 2531 scope.go:117] "RemoveContainer" containerID="552a97eab696c632f976db5d96531a3af3ad58d3e96afb9530dcd35cfc22fc56" Jul 23 13:37:08.770738 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.770720 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:37:08.861571 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.861540 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-istio-data\") pod \"29033909-c825-4062-a368-2e1c043e6a99\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " Jul 23 13:37:08.861571 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.861572 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29033909-c825-4062-a368-2e1c043e6a99-istiod-ca-cert\") pod \"29033909-c825-4062-a368-2e1c043e6a99\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " Jul 23 13:37:08.861827 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.861589 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29033909-c825-4062-a368-2e1c043e6a99-istio-podinfo\") pod \"29033909-c825-4062-a368-2e1c043e6a99\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " Jul 23 13:37:08.861827 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.861630 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-credential-socket\") pod \"29033909-c825-4062-a368-2e1c043e6a99\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " Jul 23 13:37:08.861827 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.861652 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-istio-envoy\") pod \"29033909-c825-4062-a368-2e1c043e6a99\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " Jul 23 13:37:08.861827 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.861671 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dr8lg\" (UniqueName: \"kubernetes.io/projected/29033909-c825-4062-a368-2e1c043e6a99-kube-api-access-dr8lg\") pod \"29033909-c825-4062-a368-2e1c043e6a99\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " Jul 23 13:37:08.861827 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.861696 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-workload-socket\") pod \"29033909-c825-4062-a368-2e1c043e6a99\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " Jul 23 13:37:08.861827 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.861718 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-workload-certs\") pod \"29033909-c825-4062-a368-2e1c043e6a99\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " Jul 23 13:37:08.861827 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.861755 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29033909-c825-4062-a368-2e1c043e6a99-istio-token\") pod \"29033909-c825-4062-a368-2e1c043e6a99\" (UID: \"29033909-c825-4062-a368-2e1c043e6a99\") " Jul 23 13:37:08.862174 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.861933 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "29033909-c825-4062-a368-2e1c043e6a99" (UID: "29033909-c825-4062-a368-2e1c043e6a99"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:37:08.862174 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.862005 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29033909-c825-4062-a368-2e1c043e6a99-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "29033909-c825-4062-a368-2e1c043e6a99" (UID: "29033909-c825-4062-a368-2e1c043e6a99"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:37:08.862280 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.862192 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "29033909-c825-4062-a368-2e1c043e6a99" (UID: "29033909-c825-4062-a368-2e1c043e6a99"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:37:08.862334 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.862278 2531 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29033909-c825-4062-a368-2e1c043e6a99-istiod-ca-cert\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:37:08.862334 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.862296 2531 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-credential-socket\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:37:08.862453 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.862323 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "29033909-c825-4062-a368-2e1c043e6a99" (UID: "29033909-c825-4062-a368-2e1c043e6a99"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:37:08.862537 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.862516 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-istio-data" (OuterVolumeSpecName: "istio-data") pod "29033909-c825-4062-a368-2e1c043e6a99" (UID: "29033909-c825-4062-a368-2e1c043e6a99"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:37:08.864055 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.864027 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "29033909-c825-4062-a368-2e1c043e6a99" (UID: "29033909-c825-4062-a368-2e1c043e6a99"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:37:08.864212 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.864186 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29033909-c825-4062-a368-2e1c043e6a99-kube-api-access-dr8lg" (OuterVolumeSpecName: "kube-api-access-dr8lg") pod "29033909-c825-4062-a368-2e1c043e6a99" (UID: "29033909-c825-4062-a368-2e1c043e6a99"). InnerVolumeSpecName "kube-api-access-dr8lg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:37:08.864393 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.864376 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/29033909-c825-4062-a368-2e1c043e6a99-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "29033909-c825-4062-a368-2e1c043e6a99" (UID: "29033909-c825-4062-a368-2e1c043e6a99"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 13:37:08.864535 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.864517 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29033909-c825-4062-a368-2e1c043e6a99-istio-token" (OuterVolumeSpecName: "istio-token") pod "29033909-c825-4062-a368-2e1c043e6a99" (UID: "29033909-c825-4062-a368-2e1c043e6a99"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:37:08.963621 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.963589 2531 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29033909-c825-4062-a368-2e1c043e6a99-istio-token\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:37:08.963621 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.963614 2531 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-istio-data\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:37:08.963621 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.963622 2531 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29033909-c825-4062-a368-2e1c043e6a99-istio-podinfo\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:37:08.963621 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.963630 2531 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-istio-envoy\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:37:08.963880 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.963637 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dr8lg\" (UniqueName: \"kubernetes.io/projected/29033909-c825-4062-a368-2e1c043e6a99-kube-api-access-dr8lg\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:37:08.963880 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.963646 2531 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-workload-socket\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:37:08.963880 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:08.963654 2531 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29033909-c825-4062-a368-2e1c043e6a99-workload-certs\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:37:09.753873 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:09.753792 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" event={"ID":"29033909-c825-4062-a368-2e1c043e6a99","Type":"ContainerDied","Data":"6110bb0ac57cf2afae76f5ec978fdcde84c3a25363e0ff0d800f9b59606c433c"} Jul 23 13:37:09.753873 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:09.753819 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp" Jul 23 13:37:09.753873 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:09.753851 2531 scope.go:117] "RemoveContainer" containerID="d0e87770b73cbe9914a69a18725567baefd91620568cec923498c67209e3ba17" Jul 23 13:37:09.811979 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:09.811943 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp"] Jul 23 13:37:09.820381 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:09.820329 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-6915d69d-gw-openshift-default-56d55597f5-gpcsp"] Jul 23 13:37:11.333002 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:11.332960 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29033909-c825-4062-a368-2e1c043e6a99" path="/var/lib/kubelet/pods/29033909-c825-4062-a368-2e1c043e6a99/volumes" Jul 23 13:37:12.595532 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:12.595496 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:37:20.817337 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:20.817302 2531 generic.go:356] "Generic (PLEG): container finished" podID="00268503-c98a-4f85-817a-8289827f50bf" containerID="31498a411f3a276595dcafc0dff62f0f08ac85d62dd00655aed442f8812ad6d6" exitCode=0 Jul 23 13:37:20.817887 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:20.817346 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" event={"ID":"00268503-c98a-4f85-817a-8289827f50bf","Type":"ContainerDied","Data":"31498a411f3a276595dcafc0dff62f0f08ac85d62dd00655aed442f8812ad6d6"} Jul 23 13:37:20.817887 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:20.817850 2531 scope.go:117] "RemoveContainer" containerID="31498a411f3a276595dcafc0dff62f0f08ac85d62dd00655aed442f8812ad6d6" Jul 23 13:37:20.818886 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:20.818853 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:37:20.818990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:20.818941 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:37:20.819063 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:20.818986 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:37:21.732125 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:21.732070 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:21.824526 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:21.824493 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" event={"ID":"00268503-c98a-4f85-817a-8289827f50bf","Type":"ContainerStarted","Data":"95204503d347e27110fa2b4077b0f7fd501db07d0f82a3e881612110fd34812d"} Jul 23 13:37:22.781978 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:22.781936 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:22.783114 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:22.783089 2531 patch_prober.go:28] interesting pod/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 23 13:37:22.783232 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:22.783144 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" podUID="00268503-c98a-4f85-817a-8289827f50bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 23 13:37:23.782080 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:23.782046 2531 patch_prober.go:28] interesting pod/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 23 13:37:23.782489 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:23.782106 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" podUID="00268503-c98a-4f85-817a-8289827f50bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 23 13:37:24.782612 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:24.782573 2531 patch_prober.go:28] interesting pod/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 23 13:37:24.782996 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:24.782643 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" podUID="00268503-c98a-4f85-817a-8289827f50bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 23 13:37:25.786046 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:25.786018 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:25.786464 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:25.786374 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:25.787141 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:25.787125 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:33.153761 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:33.153725 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-94afaac2-hxm8r"] Jul 23 13:37:33.154168 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:33.154125 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29033909-c825-4062-a368-2e1c043e6a99" containerName="istio-proxy" Jul 23 13:37:33.154168 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:33.154136 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="29033909-c825-4062-a368-2e1c043e6a99" containerName="istio-proxy" Jul 23 13:37:33.154168 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:33.154157 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29033909-c825-4062-a368-2e1c043e6a99" containerName="istio-proxy" Jul 23 13:37:33.154168 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:33.154162 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="29033909-c825-4062-a368-2e1c043e6a99" containerName="istio-proxy" Jul 23 13:37:33.154333 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:33.154224 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="29033909-c825-4062-a368-2e1c043e6a99" containerName="istio-proxy" Jul 23 13:37:33.154333 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:33.154232 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="29033909-c825-4062-a368-2e1c043e6a99" containerName="istio-proxy" Jul 23 13:37:33.157488 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:33.157472 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-94afaac2-hxm8r" Jul 23 13:37:33.165617 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:33.165595 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 13:37:33.165617 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:33.165595 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vp6nn\"" Jul 23 13:37:33.182990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:33.182962 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-94afaac2-hxm8r"] Jul 23 13:37:33.292713 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:33.292681 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ed435eb-40a2-43f5-9772-79e7e7095ba8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-94afaac2-hxm8r\" (UID: \"6ed435eb-40a2-43f5-9772-79e7e7095ba8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-94afaac2-hxm8r" Jul 23 13:37:33.393445 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:33.393409 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ed435eb-40a2-43f5-9772-79e7e7095ba8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-94afaac2-hxm8r\" (UID: \"6ed435eb-40a2-43f5-9772-79e7e7095ba8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-94afaac2-hxm8r" Jul 23 13:37:33.394062 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:33.394035 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ed435eb-40a2-43f5-9772-79e7e7095ba8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-94afaac2-hxm8r\" (UID: \"6ed435eb-40a2-43f5-9772-79e7e7095ba8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-94afaac2-hxm8r" Jul 23 13:37:33.467668 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:33.467545 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-94afaac2-hxm8r" Jul 23 13:37:33.614779 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:33.614752 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-94afaac2-hxm8r"] Jul 23 13:37:33.616707 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:37:33.616664 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ed435eb_40a2_43f5_9772_79e7e7095ba8.slice/crio-fec3437890531f3fb1129e96eeb918022b702e66f429b10b3b1351c2902a3beb WatchSource:0}: Error finding container fec3437890531f3fb1129e96eeb918022b702e66f429b10b3b1351c2902a3beb: Status 404 returned error can't find the container with id fec3437890531f3fb1129e96eeb918022b702e66f429b10b3b1351c2902a3beb Jul 23 13:37:33.902120 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:33.902081 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-94afaac2-hxm8r" event={"ID":"6ed435eb-40a2-43f5-9772-79e7e7095ba8","Type":"ContainerStarted","Data":"fec3437890531f3fb1129e96eeb918022b702e66f429b10b3b1351c2902a3beb"} Jul 23 13:37:34.909761 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:34.909731 2531 generic.go:356] "Generic (PLEG): container finished" podID="6ed435eb-40a2-43f5-9772-79e7e7095ba8" containerID="00ab7ab7c12176bda808c39c7c7cb5d6d707d43987324fa148c7bcdba9ff1081" exitCode=0 Jul 23 13:37:34.910143 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:34.909788 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-94afaac2-hxm8r" event={"ID":"6ed435eb-40a2-43f5-9772-79e7e7095ba8","Type":"ContainerDied","Data":"00ab7ab7c12176bda808c39c7c7cb5d6d707d43987324fa148c7bcdba9ff1081"} Jul 23 13:37:36.045575 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:36.045551 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-94afaac2-hxm8r" Jul 23 13:37:36.117304 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:36.117271 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ed435eb-40a2-43f5-9772-79e7e7095ba8-maas-api-service-ca\") pod \"6ed435eb-40a2-43f5-9772-79e7e7095ba8\" (UID: \"6ed435eb-40a2-43f5-9772-79e7e7095ba8\") " Jul 23 13:37:36.117649 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:36.117626 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ed435eb-40a2-43f5-9772-79e7e7095ba8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6ed435eb-40a2-43f5-9772-79e7e7095ba8" (UID: "6ed435eb-40a2-43f5-9772-79e7e7095ba8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:37:36.218230 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:36.218147 2531 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ed435eb-40a2-43f5-9772-79e7e7095ba8-maas-api-service-ca\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:37:36.927241 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:36.927198 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-94afaac2-hxm8r" event={"ID":"6ed435eb-40a2-43f5-9772-79e7e7095ba8","Type":"ContainerDied","Data":"fec3437890531f3fb1129e96eeb918022b702e66f429b10b3b1351c2902a3beb"} Jul 23 13:37:36.927241 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:36.927232 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-94afaac2-hxm8r" Jul 23 13:37:36.927529 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:36.927289 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fec3437890531f3fb1129e96eeb918022b702e66f429b10b3b1351c2902a3beb" Jul 23 13:37:43.364196 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:43.364169 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6m8j_a0a5f6bc-bdb7-46ae-8cb9-79edd5179817/ovn-acl-logging/0.log" Jul 23 13:37:43.367922 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:43.367901 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6m8j_a0a5f6bc-bdb7-46ae-8cb9-79edd5179817/ovn-acl-logging/0.log" Jul 23 13:37:46.690878 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:46.690839 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:37:48.203768 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:48.203738 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-94afaac2-hxm8r"] Jul 23 13:37:48.205956 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:48.205934 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-94afaac2-hxm8r"] Jul 23 13:37:49.333019 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:49.332977 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ed435eb-40a2-43f5-9772-79e7e7095ba8" path="/var/lib/kubelet/pods/6ed435eb-40a2-43f5-9772-79e7e7095ba8/volumes" Jul 23 13:37:49.463438 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:49.463404 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5"] Jul 23 13:37:49.463666 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:49.463643 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" podUID="00268503-c98a-4f85-817a-8289827f50bf" containerName="istio-proxy" containerID="cri-o://95204503d347e27110fa2b4077b0f7fd501db07d0f82a3e881612110fd34812d" gracePeriod=30 Jul 23 13:37:50.273874 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.273828 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf"] Jul 23 13:37:50.274417 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.274396 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ed435eb-40a2-43f5-9772-79e7e7095ba8" containerName="revoke-keys" Jul 23 13:37:50.274503 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.274420 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ed435eb-40a2-43f5-9772-79e7e7095ba8" containerName="revoke-keys" Jul 23 13:37:50.274549 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.274511 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ed435eb-40a2-43f5-9772-79e7e7095ba8" containerName="revoke-keys" Jul 23 13:37:50.279143 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.279123 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.282165 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.282145 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-d97728e3-openshift-default-dockercfg-kcjg4\"" Jul 23 13:37:50.294402 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.294372 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf"] Jul 23 13:37:50.352043 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.351997 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istio-podinfo\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.352501 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.352065 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istio-envoy\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.352501 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.352107 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-workload-certs\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.352501 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.352133 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istio-token\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.352501 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.352162 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-workload-socket\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.352501 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.352185 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istiod-ca-cert\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.352501 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.352269 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-credential-socket\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.352501 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.352306 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zm5m\" (UniqueName: \"kubernetes.io/projected/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-kube-api-access-7zm5m\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.352501 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.352331 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istio-data\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.453211 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.453178 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-credential-socket\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.453433 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.453226 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zm5m\" (UniqueName: \"kubernetes.io/projected/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-kube-api-access-7zm5m\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.453433 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.453259 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istio-data\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.453433 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.453291 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istio-podinfo\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.453433 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.453328 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istio-envoy\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.453433 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.453389 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-workload-certs\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.453717 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.453484 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istio-token\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.453717 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.453508 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-workload-socket\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.453717 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.453541 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istiod-ca-cert\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.453879 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.453746 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-credential-socket\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.453960 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.453936 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-workload-certs\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.454023 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.453961 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istio-data\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.454223 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.454160 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-workload-socket\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.454464 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.454249 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istiod-ca-cert\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.455893 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.455868 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istio-envoy\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.455995 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.455923 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istio-podinfo\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.464917 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.464895 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zm5m\" (UniqueName: \"kubernetes.io/projected/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-kube-api-access-7zm5m\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.466118 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.466098 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istio-token\") pod \"e2e-derive-d97728e3-openshift-default-d46c65787-kncpf\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.589434 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.589393 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:50.734964 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.734935 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf"] Jul 23 13:37:50.736801 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:37:50.736775 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod515f93f6_ddb3_4fc8_8ced_e475c2dcdb48.slice/crio-9c445ff0cfe21cf600c92f4da589c4295e83713812d967e3cf10a7481f978312 WatchSource:0}: Error finding container 9c445ff0cfe21cf600c92f4da589c4295e83713812d967e3cf10a7481f978312: Status 404 returned error can't find the container with id 9c445ff0cfe21cf600c92f4da589c4295e83713812d967e3cf10a7481f978312 Jul 23 13:37:50.739055 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.739018 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:37:50.739150 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.739104 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:37:50.739217 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:50.739150 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:37:51.007579 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:51.007547 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" event={"ID":"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48","Type":"ContainerStarted","Data":"42dd38e40b0f523dcfe0ef0572ba9b81318cd63342dd3ce550c03ccdc1e51339"} Jul 23 13:37:51.007756 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:51.007603 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" event={"ID":"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48","Type":"ContainerStarted","Data":"9c445ff0cfe21cf600c92f4da589c4295e83713812d967e3cf10a7481f978312"} Jul 23 13:37:51.030102 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:51.030048 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" podStartSLOduration=1.030031214 podStartE2EDuration="1.030031214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:37:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:37:51.028015604 +0000 UTC m=+1808.267291583" watchObservedRunningTime="2026-07-23 13:37:51.030031214 +0000 UTC m=+1808.269307190" Jul 23 13:37:51.590589 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:51.590554 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:51.595785 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:51.595757 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:52.013410 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:52.013322 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:52.014422 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:52.014399 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:37:54.713676 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.713646 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:54.787166 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.787082 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-istio-envoy\") pod \"00268503-c98a-4f85-817a-8289827f50bf\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " Jul 23 13:37:54.787166 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.787124 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00268503-c98a-4f85-817a-8289827f50bf-istio-podinfo\") pod \"00268503-c98a-4f85-817a-8289827f50bf\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " Jul 23 13:37:54.787384 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.787168 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cr6dn\" (UniqueName: \"kubernetes.io/projected/00268503-c98a-4f85-817a-8289827f50bf-kube-api-access-cr6dn\") pod \"00268503-c98a-4f85-817a-8289827f50bf\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " Jul 23 13:37:54.787384 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.787191 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00268503-c98a-4f85-817a-8289827f50bf-istio-token\") pod \"00268503-c98a-4f85-817a-8289827f50bf\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " Jul 23 13:37:54.787384 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.787227 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-workload-certs\") pod \"00268503-c98a-4f85-817a-8289827f50bf\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " Jul 23 13:37:54.787384 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.787309 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-credential-socket\") pod \"00268503-c98a-4f85-817a-8289827f50bf\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " Jul 23 13:37:54.787384 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.787337 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00268503-c98a-4f85-817a-8289827f50bf-istiod-ca-cert\") pod \"00268503-c98a-4f85-817a-8289827f50bf\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " Jul 23 13:37:54.787709 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.787438 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-workload-socket\") pod \"00268503-c98a-4f85-817a-8289827f50bf\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " Jul 23 13:37:54.787709 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.787482 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-istio-data\") pod \"00268503-c98a-4f85-817a-8289827f50bf\" (UID: \"00268503-c98a-4f85-817a-8289827f50bf\") " Jul 23 13:37:54.787709 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.787651 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "00268503-c98a-4f85-817a-8289827f50bf" (UID: "00268503-c98a-4f85-817a-8289827f50bf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:37:54.787896 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.787871 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "00268503-c98a-4f85-817a-8289827f50bf" (UID: "00268503-c98a-4f85-817a-8289827f50bf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:37:54.787957 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.787916 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00268503-c98a-4f85-817a-8289827f50bf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "00268503-c98a-4f85-817a-8289827f50bf" (UID: "00268503-c98a-4f85-817a-8289827f50bf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:37:54.788018 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.787917 2531 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-workload-certs\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:37:54.788067 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.788010 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "00268503-c98a-4f85-817a-8289827f50bf" (UID: "00268503-c98a-4f85-817a-8289827f50bf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:37:54.788873 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.788850 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-istio-data" (OuterVolumeSpecName: "istio-data") pod "00268503-c98a-4f85-817a-8289827f50bf" (UID: "00268503-c98a-4f85-817a-8289827f50bf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:37:54.789539 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.789519 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "00268503-c98a-4f85-817a-8289827f50bf" (UID: "00268503-c98a-4f85-817a-8289827f50bf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:37:54.789681 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.789664 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/00268503-c98a-4f85-817a-8289827f50bf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "00268503-c98a-4f85-817a-8289827f50bf" (UID: "00268503-c98a-4f85-817a-8289827f50bf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 13:37:54.789859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.789844 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00268503-c98a-4f85-817a-8289827f50bf-kube-api-access-cr6dn" (OuterVolumeSpecName: "kube-api-access-cr6dn") pod "00268503-c98a-4f85-817a-8289827f50bf" (UID: "00268503-c98a-4f85-817a-8289827f50bf"). InnerVolumeSpecName "kube-api-access-cr6dn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:37:54.789988 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.789971 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00268503-c98a-4f85-817a-8289827f50bf-istio-token" (OuterVolumeSpecName: "istio-token") pod "00268503-c98a-4f85-817a-8289827f50bf" (UID: "00268503-c98a-4f85-817a-8289827f50bf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:37:54.889277 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.889243 2531 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-credential-socket\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:37:54.889277 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.889272 2531 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00268503-c98a-4f85-817a-8289827f50bf-istiod-ca-cert\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:37:54.889277 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.889281 2531 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-workload-socket\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:37:54.889534 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.889288 2531 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-istio-data\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:37:54.889534 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.889296 2531 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00268503-c98a-4f85-817a-8289827f50bf-istio-envoy\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:37:54.889534 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.889303 2531 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00268503-c98a-4f85-817a-8289827f50bf-istio-podinfo\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:37:54.889534 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.889311 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cr6dn\" (UniqueName: \"kubernetes.io/projected/00268503-c98a-4f85-817a-8289827f50bf-kube-api-access-cr6dn\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:37:54.889534 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:54.889318 2531 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00268503-c98a-4f85-817a-8289827f50bf-istio-token\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:37:55.031684 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:55.031655 2531 generic.go:356] "Generic (PLEG): container finished" podID="00268503-c98a-4f85-817a-8289827f50bf" containerID="95204503d347e27110fa2b4077b0f7fd501db07d0f82a3e881612110fd34812d" exitCode=0 Jul 23 13:37:55.031859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:55.031705 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" Jul 23 13:37:55.031859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:55.031738 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" event={"ID":"00268503-c98a-4f85-817a-8289827f50bf","Type":"ContainerDied","Data":"95204503d347e27110fa2b4077b0f7fd501db07d0f82a3e881612110fd34812d"} Jul 23 13:37:55.031859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:55.031792 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5" event={"ID":"00268503-c98a-4f85-817a-8289827f50bf","Type":"ContainerDied","Data":"035d06c896715798ef42b1dc3ff61afc872c86a51d7dd02a94b38bbc2c6376c7"} Jul 23 13:37:55.031859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:55.031815 2531 scope.go:117] "RemoveContainer" containerID="95204503d347e27110fa2b4077b0f7fd501db07d0f82a3e881612110fd34812d" Jul 23 13:37:55.044784 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:55.044763 2531 scope.go:117] "RemoveContainer" containerID="31498a411f3a276595dcafc0dff62f0f08ac85d62dd00655aed442f8812ad6d6" Jul 23 13:37:55.053104 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:55.053083 2531 scope.go:117] "RemoveContainer" containerID="95204503d347e27110fa2b4077b0f7fd501db07d0f82a3e881612110fd34812d" Jul 23 13:37:55.053341 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:37:55.053324 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95204503d347e27110fa2b4077b0f7fd501db07d0f82a3e881612110fd34812d\": container with ID starting with 95204503d347e27110fa2b4077b0f7fd501db07d0f82a3e881612110fd34812d not found: ID does not exist" containerID="95204503d347e27110fa2b4077b0f7fd501db07d0f82a3e881612110fd34812d" Jul 23 13:37:55.053416 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:55.053348 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95204503d347e27110fa2b4077b0f7fd501db07d0f82a3e881612110fd34812d"} err="failed to get container status \"95204503d347e27110fa2b4077b0f7fd501db07d0f82a3e881612110fd34812d\": rpc error: code = NotFound desc = could not find container \"95204503d347e27110fa2b4077b0f7fd501db07d0f82a3e881612110fd34812d\": container with ID starting with 95204503d347e27110fa2b4077b0f7fd501db07d0f82a3e881612110fd34812d not found: ID does not exist" Jul 23 13:37:55.053416 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:55.053386 2531 scope.go:117] "RemoveContainer" containerID="31498a411f3a276595dcafc0dff62f0f08ac85d62dd00655aed442f8812ad6d6" Jul 23 13:37:55.053623 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:37:55.053606 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31498a411f3a276595dcafc0dff62f0f08ac85d62dd00655aed442f8812ad6d6\": container with ID starting with 31498a411f3a276595dcafc0dff62f0f08ac85d62dd00655aed442f8812ad6d6 not found: ID does not exist" containerID="31498a411f3a276595dcafc0dff62f0f08ac85d62dd00655aed442f8812ad6d6" Jul 23 13:37:55.053676 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:55.053627 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31498a411f3a276595dcafc0dff62f0f08ac85d62dd00655aed442f8812ad6d6"} err="failed to get container status \"31498a411f3a276595dcafc0dff62f0f08ac85d62dd00655aed442f8812ad6d6\": rpc error: code = NotFound desc = could not find container \"31498a411f3a276595dcafc0dff62f0f08ac85d62dd00655aed442f8812ad6d6\": container with ID starting with 31498a411f3a276595dcafc0dff62f0f08ac85d62dd00655aed442f8812ad6d6 not found: ID does not exist" Jul 23 13:37:55.060140 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:55.060117 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5"] Jul 23 13:37:55.067395 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:55.067371 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-94afaac2-openshift-default-54576b5885-zgxv5"] Jul 23 13:37:55.332819 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:55.332787 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00268503-c98a-4f85-817a-8289827f50bf" path="/var/lib/kubelet/pods/00268503-c98a-4f85-817a-8289827f50bf/volumes" Jul 23 13:37:57.691924 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:37:57.691884 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:38:06.100099 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:06.100066 2531 generic.go:356] "Generic (PLEG): container finished" podID="515f93f6-ddb3-4fc8-8ced-e475c2dcdb48" containerID="42dd38e40b0f523dcfe0ef0572ba9b81318cd63342dd3ce550c03ccdc1e51339" exitCode=0 Jul 23 13:38:06.100574 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:06.100149 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" event={"ID":"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48","Type":"ContainerDied","Data":"42dd38e40b0f523dcfe0ef0572ba9b81318cd63342dd3ce550c03ccdc1e51339"} Jul 23 13:38:06.100574 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:06.100566 2531 scope.go:117] "RemoveContainer" containerID="42dd38e40b0f523dcfe0ef0572ba9b81318cd63342dd3ce550c03ccdc1e51339" Jul 23 13:38:06.101673 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:06.101640 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:38:06.101751 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:06.101730 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:38:06.101788 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:06.101778 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:38:07.014003 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:07.013970 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:38:07.107489 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:07.107444 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" event={"ID":"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48","Type":"ContainerStarted","Data":"152ae3e66a8c2e0d73fce914e7cffb6e0a7239b5589574be7a7f3c41a2f0afd5"} Jul 23 13:38:07.589970 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:07.589933 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:38:07.591215 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:07.591189 2531 patch_prober.go:28] interesting pod/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 23 13:38:07.591329 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:07.591263 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" podUID="515f93f6-ddb3-4fc8-8ced-e475c2dcdb48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 23 13:38:08.590324 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:08.590289 2531 patch_prober.go:28] interesting pod/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 23 13:38:08.590713 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:08.590349 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" podUID="515f93f6-ddb3-4fc8-8ced-e475c2dcdb48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 23 13:38:09.590633 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:09.590589 2531 patch_prober.go:28] interesting pod/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 23 13:38:09.591111 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:09.590653 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" podUID="515f93f6-ddb3-4fc8-8ced-e475c2dcdb48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 23 13:38:10.590093 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:10.590053 2531 patch_prober.go:28] interesting pod/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 23 13:38:10.590281 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:10.590126 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" podUID="515f93f6-ddb3-4fc8-8ced-e475c2dcdb48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 23 13:38:11.594424 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:11.594390 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:38:11.594808 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:11.594718 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:38:11.595441 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:11.595424 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:38:17.352066 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:17.352033 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d97728e3-vtjp7"] Jul 23 13:38:17.352481 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:17.352424 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00268503-c98a-4f85-817a-8289827f50bf" containerName="istio-proxy" Jul 23 13:38:17.352481 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:17.352435 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="00268503-c98a-4f85-817a-8289827f50bf" containerName="istio-proxy" Jul 23 13:38:17.352481 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:17.352445 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00268503-c98a-4f85-817a-8289827f50bf" containerName="istio-proxy" Jul 23 13:38:17.352481 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:17.352452 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="00268503-c98a-4f85-817a-8289827f50bf" containerName="istio-proxy" Jul 23 13:38:17.352626 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:17.352535 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="00268503-c98a-4f85-817a-8289827f50bf" containerName="istio-proxy" Jul 23 13:38:17.352626 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:17.352544 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="00268503-c98a-4f85-817a-8289827f50bf" containerName="istio-proxy" Jul 23 13:38:17.355554 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:17.355538 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d97728e3-vtjp7" Jul 23 13:38:17.359137 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:17.359116 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vp6nn\"" Jul 23 13:38:17.359744 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:17.359724 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 13:38:17.377302 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:17.377272 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d97728e3-vtjp7"] Jul 23 13:38:17.514445 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:17.514403 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1d37599-19f0-423d-95b1-4f39bc7a951d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-d97728e3-vtjp7\" (UID: \"d1d37599-19f0-423d-95b1-4f39bc7a951d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d97728e3-vtjp7" Jul 23 13:38:17.615904 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:17.615815 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1d37599-19f0-423d-95b1-4f39bc7a951d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-d97728e3-vtjp7\" (UID: \"d1d37599-19f0-423d-95b1-4f39bc7a951d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d97728e3-vtjp7" Jul 23 13:38:17.616481 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:17.616455 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1d37599-19f0-423d-95b1-4f39bc7a951d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-d97728e3-vtjp7\" (UID: \"d1d37599-19f0-423d-95b1-4f39bc7a951d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d97728e3-vtjp7" Jul 23 13:38:17.666432 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:17.666400 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d97728e3-vtjp7" Jul 23 13:38:17.807271 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:17.807178 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d97728e3-vtjp7"] Jul 23 13:38:17.810097 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:38:17.810075 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1d37599_19f0_423d_95b1_4f39bc7a951d.slice/crio-18d3c311733064671bd3d130b703dac2621561f13102d5deaae34574c85e150e WatchSource:0}: Error finding container 18d3c311733064671bd3d130b703dac2621561f13102d5deaae34574c85e150e: Status 404 returned error can't find the container with id 18d3c311733064671bd3d130b703dac2621561f13102d5deaae34574c85e150e Jul 23 13:38:18.171799 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:18.171763 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d97728e3-vtjp7" event={"ID":"d1d37599-19f0-423d-95b1-4f39bc7a951d","Type":"ContainerStarted","Data":"bffdebb999fa1ffc7f59498d77c9325fdf8c085c9a5b39a41285048b63d40fea"} Jul 23 13:38:18.171958 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:18.171814 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d97728e3-vtjp7" event={"ID":"d1d37599-19f0-423d-95b1-4f39bc7a951d","Type":"ContainerStarted","Data":"18d3c311733064671bd3d130b703dac2621561f13102d5deaae34574c85e150e"} Jul 23 13:38:18.193841 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:18.193789 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d97728e3-vtjp7" podStartSLOduration=1.193772283 podStartE2EDuration="1.193772283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:38:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:38:18.190568714 +0000 UTC m=+1835.429844690" watchObservedRunningTime="2026-07-23 13:38:18.193772283 +0000 UTC m=+1835.433048299" Jul 23 13:38:19.179274 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:19.179234 2531 generic.go:356] "Generic (PLEG): container finished" podID="d1d37599-19f0-423d-95b1-4f39bc7a951d" containerID="bffdebb999fa1ffc7f59498d77c9325fdf8c085c9a5b39a41285048b63d40fea" exitCode=0 Jul 23 13:38:19.179715 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:19.179292 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d97728e3-vtjp7" event={"ID":"d1d37599-19f0-423d-95b1-4f39bc7a951d","Type":"ContainerDied","Data":"bffdebb999fa1ffc7f59498d77c9325fdf8c085c9a5b39a41285048b63d40fea"} Jul 23 13:38:20.319758 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:20.319732 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d97728e3-vtjp7" Jul 23 13:38:20.443176 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:20.443093 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1d37599-19f0-423d-95b1-4f39bc7a951d-maas-api-service-ca\") pod \"d1d37599-19f0-423d-95b1-4f39bc7a951d\" (UID: \"d1d37599-19f0-423d-95b1-4f39bc7a951d\") " Jul 23 13:38:20.443472 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:20.443452 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1d37599-19f0-423d-95b1-4f39bc7a951d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d1d37599-19f0-423d-95b1-4f39bc7a951d" (UID: "d1d37599-19f0-423d-95b1-4f39bc7a951d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:38:20.544539 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:20.544503 2531 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1d37599-19f0-423d-95b1-4f39bc7a951d-maas-api-service-ca\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:38:21.192709 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:21.192673 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d97728e3-vtjp7" Jul 23 13:38:21.192882 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:21.192713 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d97728e3-vtjp7" event={"ID":"d1d37599-19f0-423d-95b1-4f39bc7a951d","Type":"ContainerDied","Data":"18d3c311733064671bd3d130b703dac2621561f13102d5deaae34574c85e150e"} Jul 23 13:38:21.192882 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:21.192764 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="18d3c311733064671bd3d130b703dac2621561f13102d5deaae34574c85e150e" Jul 23 13:38:30.734231 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:30.734198 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:38:32.417772 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:32.417736 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d97728e3-vtjp7"] Jul 23 13:38:32.421901 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:32.421878 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d97728e3-vtjp7"] Jul 23 13:38:32.567075 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:32.567045 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf"] Jul 23 13:38:32.567347 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:32.567309 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" podUID="515f93f6-ddb3-4fc8-8ced-e475c2dcdb48" containerName="istio-proxy" containerID="cri-o://152ae3e66a8c2e0d73fce914e7cffb6e0a7239b5589574be7a7f3c41a2f0afd5" gracePeriod=30 Jul 23 13:38:33.334527 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:33.334486 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1d37599-19f0-423d-95b1-4f39bc7a951d" path="/var/lib/kubelet/pods/d1d37599-19f0-423d-95b1-4f39bc7a951d/volumes" Jul 23 13:38:37.822148 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:37.822124 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:38:37.911109 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:37.911015 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zm5m\" (UniqueName: \"kubernetes.io/projected/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-kube-api-access-7zm5m\") pod \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " Jul 23 13:38:37.911109 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:37.911096 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-workload-socket\") pod \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " Jul 23 13:38:37.911320 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:37.911122 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-credential-socket\") pod \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " Jul 23 13:38:37.911320 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:37.911168 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istio-data\") pod \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " Jul 23 13:38:37.911320 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:37.911205 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istiod-ca-cert\") pod \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " Jul 23 13:38:37.911320 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:37.911231 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istio-podinfo\") pod \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " Jul 23 13:38:37.911320 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:37.911260 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-workload-certs\") pod \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " Jul 23 13:38:37.911320 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:37.911297 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istio-token\") pod \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " Jul 23 13:38:37.911660 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:37.911322 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istio-envoy\") pod \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\" (UID: \"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48\") " Jul 23 13:38:37.911660 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:37.911478 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "515f93f6-ddb3-4fc8-8ced-e475c2dcdb48" (UID: "515f93f6-ddb3-4fc8-8ced-e475c2dcdb48"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:38:37.911660 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:37.911490 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "515f93f6-ddb3-4fc8-8ced-e475c2dcdb48" (UID: "515f93f6-ddb3-4fc8-8ced-e475c2dcdb48"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:38:37.911660 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:37.911642 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "515f93f6-ddb3-4fc8-8ced-e475c2dcdb48" (UID: "515f93f6-ddb3-4fc8-8ced-e475c2dcdb48"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:38:37.911863 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:37.911661 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "515f93f6-ddb3-4fc8-8ced-e475c2dcdb48" (UID: "515f93f6-ddb3-4fc8-8ced-e475c2dcdb48"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:38:37.911863 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:37.911759 2531 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-workload-socket\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:38:37.911863 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:37.911775 2531 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-credential-socket\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:38:37.911863 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:37.911792 2531 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istiod-ca-cert\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:38:37.911863 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:37.911810 2531 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-workload-certs\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:38:37.912204 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:37.912181 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istio-data" (OuterVolumeSpecName: "istio-data") pod "515f93f6-ddb3-4fc8-8ced-e475c2dcdb48" (UID: "515f93f6-ddb3-4fc8-8ced-e475c2dcdb48"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:38:37.913515 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:37.913484 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-kube-api-access-7zm5m" (OuterVolumeSpecName: "kube-api-access-7zm5m") pod "515f93f6-ddb3-4fc8-8ced-e475c2dcdb48" (UID: "515f93f6-ddb3-4fc8-8ced-e475c2dcdb48"). InnerVolumeSpecName "kube-api-access-7zm5m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:38:37.913624 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:37.913569 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "515f93f6-ddb3-4fc8-8ced-e475c2dcdb48" (UID: "515f93f6-ddb3-4fc8-8ced-e475c2dcdb48"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:38:37.914008 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:37.913989 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istio-token" (OuterVolumeSpecName: "istio-token") pod "515f93f6-ddb3-4fc8-8ced-e475c2dcdb48" (UID: "515f93f6-ddb3-4fc8-8ced-e475c2dcdb48"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:38:37.914008 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:37.913990 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "515f93f6-ddb3-4fc8-8ced-e475c2dcdb48" (UID: "515f93f6-ddb3-4fc8-8ced-e475c2dcdb48"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 13:38:38.012705 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:38.012671 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zm5m\" (UniqueName: \"kubernetes.io/projected/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-kube-api-access-7zm5m\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:38:38.012705 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:38.012701 2531 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istio-data\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:38:38.012705 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:38.012710 2531 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istio-podinfo\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:38:38.012705 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:38.012718 2531 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istio-token\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:38:38.012982 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:38.012725 2531 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48-istio-envoy\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:38:38.303433 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:38.303314 2531 generic.go:356] "Generic (PLEG): container finished" podID="515f93f6-ddb3-4fc8-8ced-e475c2dcdb48" containerID="152ae3e66a8c2e0d73fce914e7cffb6e0a7239b5589574be7a7f3c41a2f0afd5" exitCode=0 Jul 23 13:38:38.303433 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:38.303394 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" Jul 23 13:38:38.303433 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:38.303413 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" event={"ID":"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48","Type":"ContainerDied","Data":"152ae3e66a8c2e0d73fce914e7cffb6e0a7239b5589574be7a7f3c41a2f0afd5"} Jul 23 13:38:38.303651 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:38.303476 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf" event={"ID":"515f93f6-ddb3-4fc8-8ced-e475c2dcdb48","Type":"ContainerDied","Data":"9c445ff0cfe21cf600c92f4da589c4295e83713812d967e3cf10a7481f978312"} Jul 23 13:38:38.303651 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:38.303491 2531 scope.go:117] "RemoveContainer" containerID="152ae3e66a8c2e0d73fce914e7cffb6e0a7239b5589574be7a7f3c41a2f0afd5" Jul 23 13:38:38.313379 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:38.313338 2531 scope.go:117] "RemoveContainer" containerID="42dd38e40b0f523dcfe0ef0572ba9b81318cd63342dd3ce550c03ccdc1e51339" Jul 23 13:38:38.321793 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:38.321776 2531 scope.go:117] "RemoveContainer" containerID="152ae3e66a8c2e0d73fce914e7cffb6e0a7239b5589574be7a7f3c41a2f0afd5" Jul 23 13:38:38.322034 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:38:38.322018 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"152ae3e66a8c2e0d73fce914e7cffb6e0a7239b5589574be7a7f3c41a2f0afd5\": container with ID starting with 152ae3e66a8c2e0d73fce914e7cffb6e0a7239b5589574be7a7f3c41a2f0afd5 not found: ID does not exist" containerID="152ae3e66a8c2e0d73fce914e7cffb6e0a7239b5589574be7a7f3c41a2f0afd5" Jul 23 13:38:38.322087 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:38.322041 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"152ae3e66a8c2e0d73fce914e7cffb6e0a7239b5589574be7a7f3c41a2f0afd5"} err="failed to get container status \"152ae3e66a8c2e0d73fce914e7cffb6e0a7239b5589574be7a7f3c41a2f0afd5\": rpc error: code = NotFound desc = could not find container \"152ae3e66a8c2e0d73fce914e7cffb6e0a7239b5589574be7a7f3c41a2f0afd5\": container with ID starting with 152ae3e66a8c2e0d73fce914e7cffb6e0a7239b5589574be7a7f3c41a2f0afd5 not found: ID does not exist" Jul 23 13:38:38.322087 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:38.322057 2531 scope.go:117] "RemoveContainer" containerID="42dd38e40b0f523dcfe0ef0572ba9b81318cd63342dd3ce550c03ccdc1e51339" Jul 23 13:38:38.322251 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:38:38.322231 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42dd38e40b0f523dcfe0ef0572ba9b81318cd63342dd3ce550c03ccdc1e51339\": container with ID starting with 42dd38e40b0f523dcfe0ef0572ba9b81318cd63342dd3ce550c03ccdc1e51339 not found: ID does not exist" containerID="42dd38e40b0f523dcfe0ef0572ba9b81318cd63342dd3ce550c03ccdc1e51339" Jul 23 13:38:38.322314 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:38.322261 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42dd38e40b0f523dcfe0ef0572ba9b81318cd63342dd3ce550c03ccdc1e51339"} err="failed to get container status \"42dd38e40b0f523dcfe0ef0572ba9b81318cd63342dd3ce550c03ccdc1e51339\": rpc error: code = NotFound desc = could not find container \"42dd38e40b0f523dcfe0ef0572ba9b81318cd63342dd3ce550c03ccdc1e51339\": container with ID starting with 42dd38e40b0f523dcfe0ef0572ba9b81318cd63342dd3ce550c03ccdc1e51339 not found: ID does not exist" Jul 23 13:38:38.327314 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:38.327292 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf"] Jul 23 13:38:38.330775 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:38.330755 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-d97728e3-openshift-default-d46c65787-kncpf"] Jul 23 13:38:39.332913 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:38:39.332873 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="515f93f6-ddb3-4fc8-8ced-e475c2dcdb48" path="/var/lib/kubelet/pods/515f93f6-ddb3-4fc8-8ced-e475c2dcdb48/volumes" Jul 23 13:40:00.310178 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:00.310142 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:40:10.402244 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:10.402208 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:40:18.813039 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:18.813008 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9"] Jul 23 13:40:18.813611 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:18.813586 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="515f93f6-ddb3-4fc8-8ced-e475c2dcdb48" containerName="istio-proxy" Jul 23 13:40:18.813611 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:18.813611 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="515f93f6-ddb3-4fc8-8ced-e475c2dcdb48" containerName="istio-proxy" Jul 23 13:40:18.813766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:18.813625 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="515f93f6-ddb3-4fc8-8ced-e475c2dcdb48" containerName="istio-proxy" Jul 23 13:40:18.813766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:18.813630 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="515f93f6-ddb3-4fc8-8ced-e475c2dcdb48" containerName="istio-proxy" Jul 23 13:40:18.813766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:18.813647 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1d37599-19f0-423d-95b1-4f39bc7a951d" containerName="revoke-keys" Jul 23 13:40:18.813766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:18.813653 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1d37599-19f0-423d-95b1-4f39bc7a951d" containerName="revoke-keys" Jul 23 13:40:18.813766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:18.813711 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="515f93f6-ddb3-4fc8-8ced-e475c2dcdb48" containerName="istio-proxy" Jul 23 13:40:18.813766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:18.813720 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="515f93f6-ddb3-4fc8-8ced-e475c2dcdb48" containerName="istio-proxy" Jul 23 13:40:18.813766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:18.813728 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1d37599-19f0-423d-95b1-4f39bc7a951d" containerName="revoke-keys" Jul 23 13:40:18.816690 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:18.816671 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:18.820686 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:18.820662 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-85d6e18b-openshift-default-dockercfg-dt8fg\"" Jul 23 13:40:18.847984 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:18.847953 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9"] Jul 23 13:40:18.940696 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:18.940656 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istio-envoy\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:18.940696 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:18.940694 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-workload-socket\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:18.940928 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:18.940713 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-workload-certs\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:18.940928 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:18.940766 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8lv9\" (UniqueName: \"kubernetes.io/projected/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-kube-api-access-g8lv9\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:18.940928 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:18.940817 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istio-token\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:18.940928 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:18.940886 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istio-podinfo\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:18.940928 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:18.940910 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-credential-socket\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:18.940928 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:18.940927 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istiod-ca-cert\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:18.941118 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:18.940941 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istio-data\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:19.041602 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.041566 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istio-podinfo\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:19.041602 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.041605 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-credential-socket\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:19.041865 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.041624 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istiod-ca-cert\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:19.041865 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.041639 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istio-data\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:19.041865 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.041701 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istio-envoy\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:19.041865 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.041716 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-workload-socket\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:19.041865 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.041736 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-workload-certs\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:19.041865 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.041776 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8lv9\" (UniqueName: \"kubernetes.io/projected/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-kube-api-access-g8lv9\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:19.041865 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.041810 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istio-token\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:19.042218 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.042093 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istio-data\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:19.042218 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.042148 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-workload-socket\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:19.042330 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.042256 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-credential-socket\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:19.042429 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.042412 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istiod-ca-cert\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:19.042777 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.042753 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-workload-certs\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:19.044079 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.044057 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istio-envoy\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:19.044280 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.044262 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istio-podinfo\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:19.052721 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.052693 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istio-token\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:19.052995 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.052979 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8lv9\" (UniqueName: \"kubernetes.io/projected/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-kube-api-access-g8lv9\") pod \"e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:19.127421 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.127306 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:19.271188 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.271156 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9"] Jul 23 13:40:19.272260 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:40:19.272231 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6e849cf_d7ca_4c5f_9309_6dfbd68acaba.slice/crio-d0bf603a6ce57e30fd0b8de619af6671217ff11ad369af3685188e4cb33909cf WatchSource:0}: Error finding container d0bf603a6ce57e30fd0b8de619af6671217ff11ad369af3685188e4cb33909cf: Status 404 returned error can't find the container with id d0bf603a6ce57e30fd0b8de619af6671217ff11ad369af3685188e4cb33909cf Jul 23 13:40:19.274570 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.274537 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:40:19.274649 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.274601 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:40:19.274649 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.274632 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:40:19.862876 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.862840 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" event={"ID":"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba","Type":"ContainerStarted","Data":"f8e056a90cfb3f4738a1d8e97069f97fea2c96c99b6c3cba55dafed4f8c1391f"} Jul 23 13:40:19.863317 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.862888 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" event={"ID":"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba","Type":"ContainerStarted","Data":"d0bf603a6ce57e30fd0b8de619af6671217ff11ad369af3685188e4cb33909cf"} Jul 23 13:40:19.896697 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:19.896644 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" podStartSLOduration=1.896628705 podStartE2EDuration="1.896628705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:40:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:40:19.894027718 +0000 UTC m=+1957.133303694" watchObservedRunningTime="2026-07-23 13:40:19.896628705 +0000 UTC m=+1957.135904681" Jul 23 13:40:20.128436 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:20.128320 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:20.133797 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:20.133769 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:20.869827 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:20.869799 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:20.870949 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:20.870927 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:22.889496 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:22.889459 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:40:32.205527 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:32.205490 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:40:33.669379 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:33.669261 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85d6e18b-m9psx"] Jul 23 13:40:33.673126 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:33.673110 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85d6e18b-m9psx" Jul 23 13:40:33.675398 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:33.675343 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 13:40:33.675398 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:33.675345 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vp6nn\"" Jul 23 13:40:33.682734 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:33.682709 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85d6e18b-m9psx"] Jul 23 13:40:33.786638 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:33.786603 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2db42e99-cf91-4bd8-bfaa-d618cd7a7e62-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-85d6e18b-m9psx\" (UID: \"2db42e99-cf91-4bd8-bfaa-d618cd7a7e62\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85d6e18b-m9psx" Jul 23 13:40:33.887376 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:33.887312 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2db42e99-cf91-4bd8-bfaa-d618cd7a7e62-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-85d6e18b-m9psx\" (UID: \"2db42e99-cf91-4bd8-bfaa-d618cd7a7e62\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85d6e18b-m9psx" Jul 23 13:40:33.888202 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:33.888178 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2db42e99-cf91-4bd8-bfaa-d618cd7a7e62-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-85d6e18b-m9psx\" (UID: \"2db42e99-cf91-4bd8-bfaa-d618cd7a7e62\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85d6e18b-m9psx" Jul 23 13:40:33.987343 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:33.987260 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85d6e18b-m9psx" Jul 23 13:40:34.149192 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:34.140105 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85d6e18b-m9psx"] Jul 23 13:40:34.953114 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:34.953080 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85d6e18b-m9psx" event={"ID":"2db42e99-cf91-4bd8-bfaa-d618cd7a7e62","Type":"ContainerStarted","Data":"d8566dfca112f7e020d3411a0563585d3ba99788e59df697487db31909643469"} Jul 23 13:40:34.953546 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:34.953134 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85d6e18b-m9psx" event={"ID":"2db42e99-cf91-4bd8-bfaa-d618cd7a7e62","Type":"ContainerStarted","Data":"5847a4bb3c890d5ea504bef07b025201bfe136def851d15012d592342156bb9d"} Jul 23 13:40:34.971601 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:34.971550 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85d6e18b-m9psx" podStartSLOduration=1.9715334979999999 podStartE2EDuration="1.971533498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:40:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:40:34.968600885 +0000 UTC m=+1972.207876860" watchObservedRunningTime="2026-07-23 13:40:34.971533498 +0000 UTC m=+1972.210809474" Jul 23 13:40:35.961280 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:35.961243 2531 generic.go:356] "Generic (PLEG): container finished" podID="2db42e99-cf91-4bd8-bfaa-d618cd7a7e62" containerID="d8566dfca112f7e020d3411a0563585d3ba99788e59df697487db31909643469" exitCode=0 Jul 23 13:40:35.961749 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:35.961340 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85d6e18b-m9psx" event={"ID":"2db42e99-cf91-4bd8-bfaa-d618cd7a7e62","Type":"ContainerDied","Data":"d8566dfca112f7e020d3411a0563585d3ba99788e59df697487db31909643469"} Jul 23 13:40:36.117502 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:36.117455 2531 patch_prober.go:28] interesting pod/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": read tcp 10.134.0.2:53094->10.134.0.54:15021: read: connection reset by peer" start-of-body= Jul 23 13:40:36.117685 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:36.117526 2531 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" podUID="d6e849cf-d7ca-4c5f-9309-6dfbd68acaba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": read tcp 10.134.0.2:53094->10.134.0.54:15021: read: connection reset by peer" Jul 23 13:40:36.968608 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:36.968574 2531 generic.go:356] "Generic (PLEG): container finished" podID="d6e849cf-d7ca-4c5f-9309-6dfbd68acaba" containerID="f8e056a90cfb3f4738a1d8e97069f97fea2c96c99b6c3cba55dafed4f8c1391f" exitCode=0 Jul 23 13:40:36.968990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:36.968652 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" event={"ID":"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba","Type":"ContainerDied","Data":"f8e056a90cfb3f4738a1d8e97069f97fea2c96c99b6c3cba55dafed4f8c1391f"} Jul 23 13:40:36.969295 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:36.969150 2531 scope.go:117] "RemoveContainer" containerID="f8e056a90cfb3f4738a1d8e97069f97fea2c96c99b6c3cba55dafed4f8c1391f" Jul 23 13:40:36.970496 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:36.970461 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:40:36.970585 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:36.970567 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:40:36.970637 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:36.970619 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:40:37.112590 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:37.112562 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85d6e18b-m9psx" Jul 23 13:40:37.120009 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:37.119983 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2db42e99-cf91-4bd8-bfaa-d618cd7a7e62-maas-api-service-ca\") pod \"2db42e99-cf91-4bd8-bfaa-d618cd7a7e62\" (UID: \"2db42e99-cf91-4bd8-bfaa-d618cd7a7e62\") " Jul 23 13:40:37.120315 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:37.120293 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2db42e99-cf91-4bd8-bfaa-d618cd7a7e62-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2db42e99-cf91-4bd8-bfaa-d618cd7a7e62" (UID: "2db42e99-cf91-4bd8-bfaa-d618cd7a7e62"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:40:37.221030 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:37.220947 2531 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2db42e99-cf91-4bd8-bfaa-d618cd7a7e62-maas-api-service-ca\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:40:37.976250 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:37.976214 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" event={"ID":"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba","Type":"ContainerStarted","Data":"ce2f3f512f0eb8d924b3eb91d0a08fa9dc676f05e652a9b03b893d390be3e3af"} Jul 23 13:40:37.977522 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:37.977501 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85d6e18b-m9psx" Jul 23 13:40:37.977650 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:37.977519 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85d6e18b-m9psx" event={"ID":"2db42e99-cf91-4bd8-bfaa-d618cd7a7e62","Type":"ContainerDied","Data":"5847a4bb3c890d5ea504bef07b025201bfe136def851d15012d592342156bb9d"} Jul 23 13:40:37.977650 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:37.977562 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5847a4bb3c890d5ea504bef07b025201bfe136def851d15012d592342156bb9d" Jul 23 13:40:38.127676 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:38.127639 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:38.127841 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:38.127790 2531 patch_prober.go:28] interesting pod/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 23 13:40:38.127841 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:38.127829 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" podUID="d6e849cf-d7ca-4c5f-9309-6dfbd68acaba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 23 13:40:39.128371 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:39.128326 2531 patch_prober.go:28] interesting pod/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 23 13:40:39.128768 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:39.128417 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" podUID="d6e849cf-d7ca-4c5f-9309-6dfbd68acaba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 23 13:40:40.128673 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:40.128632 2531 patch_prober.go:28] interesting pod/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 23 13:40:40.129157 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:40.128687 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" podUID="d6e849cf-d7ca-4c5f-9309-6dfbd68acaba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 23 13:40:41.128286 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:41.128244 2531 patch_prober.go:28] interesting pod/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 23 13:40:41.128522 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:41.128306 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" podUID="d6e849cf-d7ca-4c5f-9309-6dfbd68acaba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 23 13:40:42.131918 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:42.131887 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:42.132408 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:42.132159 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:42.132999 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:42.132984 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:40:47.494603 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:47.494568 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:40:48.762210 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:48.762176 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85d6e18b-m9psx"] Jul 23 13:40:48.764771 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:48.764744 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-85d6e18b-m9psx"] Jul 23 13:40:49.332401 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:49.332371 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2db42e99-cf91-4bd8-bfaa-d618cd7a7e62" path="/var/lib/kubelet/pods/2db42e99-cf91-4bd8-bfaa-d618cd7a7e62/volumes" Jul 23 13:40:55.333033 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:55.333004 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9"] Jul 23 13:40:55.333460 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:55.333219 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" podUID="d6e849cf-d7ca-4c5f-9309-6dfbd68acaba" containerName="istio-proxy" containerID="cri-o://ce2f3f512f0eb8d924b3eb91d0a08fa9dc676f05e652a9b03b893d390be3e3af" gracePeriod=30 Jul 23 13:40:57.135150 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:57.135108 2531 patch_prober.go:28] interesting pod/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 13:40:57.135560 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:40:57.135163 2531 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" podUID="d6e849cf-d7ca-4c5f-9309-6dfbd68acaba" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 13:41:00.591573 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.591545 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:41:00.641576 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.641497 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-workload-certs\") pod \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " Jul 23 13:41:00.641576 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.641529 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-credential-socket\") pod \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " Jul 23 13:41:00.641773 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.641604 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istio-envoy\") pod \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " Jul 23 13:41:00.641773 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.641636 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istio-podinfo\") pod \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " Jul 23 13:41:00.641773 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.641660 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istiod-ca-cert\") pod \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " Jul 23 13:41:00.641773 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.641691 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istio-data\") pod \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " Jul 23 13:41:00.641773 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.641718 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8lv9\" (UniqueName: \"kubernetes.io/projected/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-kube-api-access-g8lv9\") pod \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " Jul 23 13:41:00.641773 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.641753 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istio-token\") pod \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " Jul 23 13:41:00.642110 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.641783 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-workload-socket\") pod \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\" (UID: \"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba\") " Jul 23 13:41:00.642110 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.641802 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d6e849cf-d7ca-4c5f-9309-6dfbd68acaba" (UID: "d6e849cf-d7ca-4c5f-9309-6dfbd68acaba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:41:00.642110 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.641890 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d6e849cf-d7ca-4c5f-9309-6dfbd68acaba" (UID: "d6e849cf-d7ca-4c5f-9309-6dfbd68acaba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:41:00.642110 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.642080 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d6e849cf-d7ca-4c5f-9309-6dfbd68acaba" (UID: "d6e849cf-d7ca-4c5f-9309-6dfbd68acaba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:41:00.642274 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.642110 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d6e849cf-d7ca-4c5f-9309-6dfbd68acaba" (UID: "d6e849cf-d7ca-4c5f-9309-6dfbd68acaba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:41:00.642274 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.642197 2531 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-workload-socket\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:41:00.642274 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.642212 2531 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-workload-certs\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:41:00.642274 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.642224 2531 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-credential-socket\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:41:00.642274 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.642236 2531 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istiod-ca-cert\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:41:00.642808 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.642745 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istio-data" (OuterVolumeSpecName: "istio-data") pod "d6e849cf-d7ca-4c5f-9309-6dfbd68acaba" (UID: "d6e849cf-d7ca-4c5f-9309-6dfbd68acaba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:41:00.644001 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.643973 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d6e849cf-d7ca-4c5f-9309-6dfbd68acaba" (UID: "d6e849cf-d7ca-4c5f-9309-6dfbd68acaba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 13:41:00.644245 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.644228 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-kube-api-access-g8lv9" (OuterVolumeSpecName: "kube-api-access-g8lv9") pod "d6e849cf-d7ca-4c5f-9309-6dfbd68acaba" (UID: "d6e849cf-d7ca-4c5f-9309-6dfbd68acaba"). InnerVolumeSpecName "kube-api-access-g8lv9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:41:00.644503 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.644475 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d6e849cf-d7ca-4c5f-9309-6dfbd68acaba" (UID: "d6e849cf-d7ca-4c5f-9309-6dfbd68acaba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:41:00.644564 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.644497 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istio-token" (OuterVolumeSpecName: "istio-token") pod "d6e849cf-d7ca-4c5f-9309-6dfbd68acaba" (UID: "d6e849cf-d7ca-4c5f-9309-6dfbd68acaba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:41:00.743186 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.743151 2531 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istio-data\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:41:00.743186 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.743180 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8lv9\" (UniqueName: \"kubernetes.io/projected/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-kube-api-access-g8lv9\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:41:00.743186 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.743191 2531 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istio-token\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:41:00.743453 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.743199 2531 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istio-envoy\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:41:00.743453 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:00.743208 2531 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba-istio-podinfo\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:41:01.106098 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:01.106059 2531 generic.go:356] "Generic (PLEG): container finished" podID="d6e849cf-d7ca-4c5f-9309-6dfbd68acaba" containerID="ce2f3f512f0eb8d924b3eb91d0a08fa9dc676f05e652a9b03b893d390be3e3af" exitCode=0 Jul 23 13:41:01.106098 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:01.106097 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" event={"ID":"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba","Type":"ContainerDied","Data":"ce2f3f512f0eb8d924b3eb91d0a08fa9dc676f05e652a9b03b893d390be3e3af"} Jul 23 13:41:01.106385 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:01.106109 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" Jul 23 13:41:01.106385 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:01.106153 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9" event={"ID":"d6e849cf-d7ca-4c5f-9309-6dfbd68acaba","Type":"ContainerDied","Data":"d0bf603a6ce57e30fd0b8de619af6671217ff11ad369af3685188e4cb33909cf"} Jul 23 13:41:01.106385 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:01.106181 2531 scope.go:117] "RemoveContainer" containerID="ce2f3f512f0eb8d924b3eb91d0a08fa9dc676f05e652a9b03b893d390be3e3af" Jul 23 13:41:01.116706 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:01.116685 2531 scope.go:117] "RemoveContainer" containerID="f8e056a90cfb3f4738a1d8e97069f97fea2c96c99b6c3cba55dafed4f8c1391f" Jul 23 13:41:01.126932 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:01.126904 2531 scope.go:117] "RemoveContainer" containerID="ce2f3f512f0eb8d924b3eb91d0a08fa9dc676f05e652a9b03b893d390be3e3af" Jul 23 13:41:01.127239 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:41:01.127217 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce2f3f512f0eb8d924b3eb91d0a08fa9dc676f05e652a9b03b893d390be3e3af\": container with ID starting with ce2f3f512f0eb8d924b3eb91d0a08fa9dc676f05e652a9b03b893d390be3e3af not found: ID does not exist" containerID="ce2f3f512f0eb8d924b3eb91d0a08fa9dc676f05e652a9b03b893d390be3e3af" Jul 23 13:41:01.127346 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:01.127251 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce2f3f512f0eb8d924b3eb91d0a08fa9dc676f05e652a9b03b893d390be3e3af"} err="failed to get container status \"ce2f3f512f0eb8d924b3eb91d0a08fa9dc676f05e652a9b03b893d390be3e3af\": rpc error: code = NotFound desc = could not find container \"ce2f3f512f0eb8d924b3eb91d0a08fa9dc676f05e652a9b03b893d390be3e3af\": container with ID starting with ce2f3f512f0eb8d924b3eb91d0a08fa9dc676f05e652a9b03b893d390be3e3af not found: ID does not exist" Jul 23 13:41:01.127346 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:01.127269 2531 scope.go:117] "RemoveContainer" containerID="f8e056a90cfb3f4738a1d8e97069f97fea2c96c99b6c3cba55dafed4f8c1391f" Jul 23 13:41:01.127556 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:41:01.127540 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8e056a90cfb3f4738a1d8e97069f97fea2c96c99b6c3cba55dafed4f8c1391f\": container with ID starting with f8e056a90cfb3f4738a1d8e97069f97fea2c96c99b6c3cba55dafed4f8c1391f not found: ID does not exist" containerID="f8e056a90cfb3f4738a1d8e97069f97fea2c96c99b6c3cba55dafed4f8c1391f" Jul 23 13:41:01.127599 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:01.127564 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8e056a90cfb3f4738a1d8e97069f97fea2c96c99b6c3cba55dafed4f8c1391f"} err="failed to get container status \"f8e056a90cfb3f4738a1d8e97069f97fea2c96c99b6c3cba55dafed4f8c1391f\": rpc error: code = NotFound desc = could not find container \"f8e056a90cfb3f4738a1d8e97069f97fea2c96c99b6c3cba55dafed4f8c1391f\": container with ID starting with f8e056a90cfb3f4738a1d8e97069f97fea2c96c99b6c3cba55dafed4f8c1391f not found: ID does not exist" Jul 23 13:41:01.129297 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:01.129274 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9"] Jul 23 13:41:01.134392 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:01.134371 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-85d6e18b-openshift-default-6c8cff4555-xf9g9"] Jul 23 13:41:01.333664 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:01.333634 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6e849cf-d7ca-4c5f-9309-6dfbd68acaba" path="/var/lib/kubelet/pods/d6e849cf-d7ca-4c5f-9309-6dfbd68acaba/volumes" Jul 23 13:41:15.356538 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.356495 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm"] Jul 23 13:41:15.357183 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.357160 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6e849cf-d7ca-4c5f-9309-6dfbd68acaba" containerName="istio-proxy" Jul 23 13:41:15.357183 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.357184 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6e849cf-d7ca-4c5f-9309-6dfbd68acaba" containerName="istio-proxy" Jul 23 13:41:15.357327 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.357214 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6e849cf-d7ca-4c5f-9309-6dfbd68acaba" containerName="istio-proxy" Jul 23 13:41:15.357327 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.357223 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6e849cf-d7ca-4c5f-9309-6dfbd68acaba" containerName="istio-proxy" Jul 23 13:41:15.357327 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.357244 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2db42e99-cf91-4bd8-bfaa-d618cd7a7e62" containerName="revoke-keys" Jul 23 13:41:15.357327 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.357253 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="2db42e99-cf91-4bd8-bfaa-d618cd7a7e62" containerName="revoke-keys" Jul 23 13:41:15.357579 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.357405 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6e849cf-d7ca-4c5f-9309-6dfbd68acaba" containerName="istio-proxy" Jul 23 13:41:15.357579 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.357425 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="2db42e99-cf91-4bd8-bfaa-d618cd7a7e62" containerName="revoke-keys" Jul 23 13:41:15.357579 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.357433 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6e849cf-d7ca-4c5f-9309-6dfbd68acaba" containerName="istio-proxy" Jul 23 13:41:15.360459 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.360440 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.363009 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.362989 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-da798329-openshift-default-dockercfg-qsbqz\"" Jul 23 13:41:15.381613 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.381581 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm"] Jul 23 13:41:15.493643 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.493598 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istio-envoy\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.493854 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.493655 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-workload-certs\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.493854 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.493814 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istio-data\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.493961 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.493866 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-credential-socket\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.493961 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.493921 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istio-token\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.494053 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.493990 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2tbc\" (UniqueName: \"kubernetes.io/projected/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-kube-api-access-m2tbc\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.494053 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.494019 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-workload-socket\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.494053 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.494042 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istiod-ca-cert\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.494195 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.494070 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istio-podinfo\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.594687 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.594631 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-credential-socket\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.594687 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.594698 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istio-token\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.594956 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.594737 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2tbc\" (UniqueName: \"kubernetes.io/projected/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-kube-api-access-m2tbc\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.594956 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.594761 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-workload-socket\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.594956 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.594783 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istiod-ca-cert\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.594956 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.594814 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istio-podinfo\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.594956 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.594848 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istio-envoy\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.594956 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.594909 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-workload-certs\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.595275 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.595010 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istio-data\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.595275 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.595106 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-credential-socket\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.595275 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.595169 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-workload-socket\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.595485 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.595335 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istio-data\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.595485 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.595423 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-workload-certs\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.595590 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.595484 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istiod-ca-cert\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.597367 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.597328 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istio-envoy\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.597583 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.597565 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istio-podinfo\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.619004 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.618943 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istio-token\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.620679 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.620653 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2tbc\" (UniqueName: \"kubernetes.io/projected/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-kube-api-access-m2tbc\") pod \"e2e-mt-da798329-openshift-default-857d8c474c-bbnvm\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.670596 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.670553 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:15.821249 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.821226 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm"] Jul 23 13:41:15.822584 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:41:15.822552 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4a6d27e_89ac_4174_9c8e_ffa9c9ff8429.slice/crio-48d9bf7f1859b2aa633279fb5c0191579c32dcf073109deeefd405319269981c WatchSource:0}: Error finding container 48d9bf7f1859b2aa633279fb5c0191579c32dcf073109deeefd405319269981c: Status 404 returned error can't find the container with id 48d9bf7f1859b2aa633279fb5c0191579c32dcf073109deeefd405319269981c Jul 23 13:41:15.824677 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.824647 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:41:15.824767 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.824711 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:41:15.824767 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:15.824738 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:41:16.198460 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:16.198349 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" event={"ID":"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429","Type":"ContainerStarted","Data":"58be17aeadf7e13f7e7c05ddc6a278a5f3daab740f8fc89e66a6f2865d4747d8"} Jul 23 13:41:16.198615 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:16.198483 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" event={"ID":"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429","Type":"ContainerStarted","Data":"48d9bf7f1859b2aa633279fb5c0191579c32dcf073109deeefd405319269981c"} Jul 23 13:41:16.223210 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:16.223156 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" podStartSLOduration=1.223141243 podStartE2EDuration="1.223141243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:41:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:41:16.220818196 +0000 UTC m=+2013.460094174" watchObservedRunningTime="2026-07-23 13:41:16.223141243 +0000 UTC m=+2013.462417268" Jul 23 13:41:16.671654 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:16.671608 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:16.676574 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:16.676549 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:17.204405 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:17.204372 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:17.205338 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:17.205316 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:25.195097 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:25.195065 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:41:33.200244 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:33.200207 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:41:35.313416 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:35.313386 2531 generic.go:356] "Generic (PLEG): container finished" podID="b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429" containerID="58be17aeadf7e13f7e7c05ddc6a278a5f3daab740f8fc89e66a6f2865d4747d8" exitCode=0 Jul 23 13:41:35.313806 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:35.313440 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" event={"ID":"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429","Type":"ContainerDied","Data":"58be17aeadf7e13f7e7c05ddc6a278a5f3daab740f8fc89e66a6f2865d4747d8"} Jul 23 13:41:35.313869 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:35.313855 2531 scope.go:117] "RemoveContainer" containerID="58be17aeadf7e13f7e7c05ddc6a278a5f3daab740f8fc89e66a6f2865d4747d8" Jul 23 13:41:35.314891 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:35.314865 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:41:35.314952 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:35.314932 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:41:35.314992 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:35.314963 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:41:36.321138 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:36.321103 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" event={"ID":"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429","Type":"ContainerStarted","Data":"61a9c7ca3ab5ccb10c68736f954eb158afe827f0f0a68e868055883308eb2973"} Jul 23 13:41:36.671785 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:36.671689 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:36.671933 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:36.671903 2531 patch_prober.go:28] interesting pod/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 23 13:41:36.671978 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:36.671952 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" podUID="b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 23 13:41:37.671685 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:37.671655 2531 patch_prober.go:28] interesting pod/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 23 13:41:37.672098 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:37.671722 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" podUID="b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 23 13:41:38.671959 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:38.671920 2531 patch_prober.go:28] interesting pod/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 23 13:41:38.672341 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:38.671990 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" podUID="b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 23 13:41:39.670962 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:39.670929 2531 patch_prober.go:28] interesting pod/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 23 13:41:39.671143 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:39.670981 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" podUID="b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 23 13:41:40.674474 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:40.674443 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:40.674844 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:40.674717 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:41:40.675490 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:41:40.675474 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:42:02.902380 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:02.902311 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:42:06.616284 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:06.616202 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-da798329-qjgcf"] Jul 23 13:42:06.618881 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:06.618864 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-da798329-qjgcf" Jul 23 13:42:06.621603 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:06.621579 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vp6nn\"" Jul 23 13:42:06.622035 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:06.622021 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 13:42:06.631559 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:06.631535 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-da798329-qjgcf"] Jul 23 13:42:06.679880 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:06.679839 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e84652c-a370-4c77-8c7b-c33800381c7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-da798329-qjgcf\" (UID: \"2e84652c-a370-4c77-8c7b-c33800381c7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-da798329-qjgcf" Jul 23 13:42:06.780822 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:06.780773 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e84652c-a370-4c77-8c7b-c33800381c7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-da798329-qjgcf\" (UID: \"2e84652c-a370-4c77-8c7b-c33800381c7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-da798329-qjgcf" Jul 23 13:42:06.781471 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:06.781448 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e84652c-a370-4c77-8c7b-c33800381c7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-da798329-qjgcf\" (UID: \"2e84652c-a370-4c77-8c7b-c33800381c7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-da798329-qjgcf" Jul 23 13:42:06.929025 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:06.928932 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-da798329-qjgcf" Jul 23 13:42:07.273003 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:07.272978 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-da798329-qjgcf"] Jul 23 13:42:07.274644 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:42:07.274614 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e84652c_a370_4c77_8c7b_c33800381c7d.slice/crio-ba4939a8a34e5acf5fd921a89810f86f22399e473ed59f61f92e18e24b61e454 WatchSource:0}: Error finding container ba4939a8a34e5acf5fd921a89810f86f22399e473ed59f61f92e18e24b61e454: Status 404 returned error can't find the container with id ba4939a8a34e5acf5fd921a89810f86f22399e473ed59f61f92e18e24b61e454 Jul 23 13:42:07.505766 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:07.505734 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-da798329-qjgcf" event={"ID":"2e84652c-a370-4c77-8c7b-c33800381c7d","Type":"ContainerStarted","Data":"ba4939a8a34e5acf5fd921a89810f86f22399e473ed59f61f92e18e24b61e454"} Jul 23 13:42:08.513045 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:08.513006 2531 generic.go:356] "Generic (PLEG): container finished" podID="2e84652c-a370-4c77-8c7b-c33800381c7d" containerID="ea5e7a780e50754509926f18843a1f43be021298dfa627f03bfab6040a2530a9" exitCode=0 Jul 23 13:42:08.513474 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:08.513091 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-da798329-qjgcf" event={"ID":"2e84652c-a370-4c77-8c7b-c33800381c7d","Type":"ContainerDied","Data":"ea5e7a780e50754509926f18843a1f43be021298dfa627f03bfab6040a2530a9"} Jul 23 13:42:09.644326 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:09.644303 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-da798329-qjgcf" Jul 23 13:42:09.710475 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:09.710442 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e84652c-a370-4c77-8c7b-c33800381c7d-maas-api-service-ca\") pod \"2e84652c-a370-4c77-8c7b-c33800381c7d\" (UID: \"2e84652c-a370-4c77-8c7b-c33800381c7d\") " Jul 23 13:42:09.710826 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:09.710807 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e84652c-a370-4c77-8c7b-c33800381c7d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2e84652c-a370-4c77-8c7b-c33800381c7d" (UID: "2e84652c-a370-4c77-8c7b-c33800381c7d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:42:09.811318 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:09.811290 2531 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e84652c-a370-4c77-8c7b-c33800381c7d-maas-api-service-ca\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:42:10.524616 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:10.524583 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-da798329-qjgcf" Jul 23 13:42:10.524791 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:10.524613 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-da798329-qjgcf" event={"ID":"2e84652c-a370-4c77-8c7b-c33800381c7d","Type":"ContainerDied","Data":"ba4939a8a34e5acf5fd921a89810f86f22399e473ed59f61f92e18e24b61e454"} Jul 23 13:42:10.524791 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:10.524661 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba4939a8a34e5acf5fd921a89810f86f22399e473ed59f61f92e18e24b61e454" Jul 23 13:42:20.597237 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:20.597196 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:42:26.686980 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:26.686940 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-da798329-qjgcf"] Jul 23 13:42:26.689938 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:26.689912 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-da798329-qjgcf"] Jul 23 13:42:27.332130 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:27.332088 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e84652c-a370-4c77-8c7b-c33800381c7d" path="/var/lib/kubelet/pods/2e84652c-a370-4c77-8c7b-c33800381c7d/volumes" Jul 23 13:42:31.136756 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:31.136712 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:42:43.400740 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:43.400712 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6m8j_a0a5f6bc-bdb7-46ae-8cb9-79edd5179817/ovn-acl-logging/0.log" Jul 23 13:42:43.406078 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:43.406055 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6m8j_a0a5f6bc-bdb7-46ae-8cb9-79edd5179817/ovn-acl-logging/0.log" Jul 23 13:42:43.470235 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:42:43.470204 2531 scope.go:117] "RemoveContainer" containerID="29434c086890a29f3c42fc9ca5df2a8bfa5b44d964e24b5a1da50e1eba29f150" Jul 23 13:43:17.595774 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:17.595736 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm"] Jul 23 13:43:17.596285 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:17.596027 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" podUID="b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429" containerName="istio-proxy" containerID="cri-o://61a9c7ca3ab5ccb10c68736f954eb158afe827f0f0a68e868055883308eb2973" gracePeriod=30 Jul 23 13:43:22.851992 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.851967 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:43:22.928753 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.928671 2531 generic.go:356] "Generic (PLEG): container finished" podID="b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429" containerID="61a9c7ca3ab5ccb10c68736f954eb158afe827f0f0a68e868055883308eb2973" exitCode=0 Jul 23 13:43:22.928753 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.928722 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" Jul 23 13:43:22.928926 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.928754 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" event={"ID":"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429","Type":"ContainerDied","Data":"61a9c7ca3ab5ccb10c68736f954eb158afe827f0f0a68e868055883308eb2973"} Jul 23 13:43:22.928926 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.928809 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm" event={"ID":"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429","Type":"ContainerDied","Data":"48d9bf7f1859b2aa633279fb5c0191579c32dcf073109deeefd405319269981c"} Jul 23 13:43:22.928926 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.928840 2531 scope.go:117] "RemoveContainer" containerID="61a9c7ca3ab5ccb10c68736f954eb158afe827f0f0a68e868055883308eb2973" Jul 23 13:43:22.940887 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.940868 2531 scope.go:117] "RemoveContainer" containerID="58be17aeadf7e13f7e7c05ddc6a278a5f3daab740f8fc89e66a6f2865d4747d8" Jul 23 13:43:22.949463 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.949437 2531 scope.go:117] "RemoveContainer" containerID="61a9c7ca3ab5ccb10c68736f954eb158afe827f0f0a68e868055883308eb2973" Jul 23 13:43:22.949725 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:43:22.949704 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61a9c7ca3ab5ccb10c68736f954eb158afe827f0f0a68e868055883308eb2973\": container with ID starting with 61a9c7ca3ab5ccb10c68736f954eb158afe827f0f0a68e868055883308eb2973 not found: ID does not exist" containerID="61a9c7ca3ab5ccb10c68736f954eb158afe827f0f0a68e868055883308eb2973" Jul 23 13:43:22.949793 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.949734 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61a9c7ca3ab5ccb10c68736f954eb158afe827f0f0a68e868055883308eb2973"} err="failed to get container status \"61a9c7ca3ab5ccb10c68736f954eb158afe827f0f0a68e868055883308eb2973\": rpc error: code = NotFound desc = could not find container \"61a9c7ca3ab5ccb10c68736f954eb158afe827f0f0a68e868055883308eb2973\": container with ID starting with 61a9c7ca3ab5ccb10c68736f954eb158afe827f0f0a68e868055883308eb2973 not found: ID does not exist" Jul 23 13:43:22.949793 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.949752 2531 scope.go:117] "RemoveContainer" containerID="58be17aeadf7e13f7e7c05ddc6a278a5f3daab740f8fc89e66a6f2865d4747d8" Jul 23 13:43:22.949997 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:43:22.949981 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58be17aeadf7e13f7e7c05ddc6a278a5f3daab740f8fc89e66a6f2865d4747d8\": container with ID starting with 58be17aeadf7e13f7e7c05ddc6a278a5f3daab740f8fc89e66a6f2865d4747d8 not found: ID does not exist" containerID="58be17aeadf7e13f7e7c05ddc6a278a5f3daab740f8fc89e66a6f2865d4747d8" Jul 23 13:43:22.950040 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.950005 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58be17aeadf7e13f7e7c05ddc6a278a5f3daab740f8fc89e66a6f2865d4747d8"} err="failed to get container status \"58be17aeadf7e13f7e7c05ddc6a278a5f3daab740f8fc89e66a6f2865d4747d8\": rpc error: code = NotFound desc = could not find container \"58be17aeadf7e13f7e7c05ddc6a278a5f3daab740f8fc89e66a6f2865d4747d8\": container with ID starting with 58be17aeadf7e13f7e7c05ddc6a278a5f3daab740f8fc89e66a6f2865d4747d8 not found: ID does not exist" Jul 23 13:43:22.988406 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.988346 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istio-token\") pod \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " Jul 23 13:43:22.988564 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.988454 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istiod-ca-cert\") pod \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " Jul 23 13:43:22.988564 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.988486 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istio-envoy\") pod \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " Jul 23 13:43:22.988564 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.988518 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-workload-certs\") pod \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " Jul 23 13:43:22.988726 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.988667 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istio-data\") pod \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " Jul 23 13:43:22.988783 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.988742 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2tbc\" (UniqueName: \"kubernetes.io/projected/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-kube-api-access-m2tbc\") pod \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " Jul 23 13:43:22.988843 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.988782 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429" (UID: "b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:43:22.988843 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.988812 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-credential-socket\") pod \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " Jul 23 13:43:22.988843 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.988838 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-workload-socket\") pod \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " Jul 23 13:43:22.989003 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.988872 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istio-podinfo\") pod \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\" (UID: \"b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429\") " Jul 23 13:43:22.989003 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.988916 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429" (UID: "b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:43:22.989138 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.989107 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429" (UID: "b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:43:22.989398 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.989189 2531 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-workload-certs\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:43:22.989652 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.989411 2531 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istiod-ca-cert\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:43:22.989652 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.989228 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429" (UID: "b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:43:22.989652 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.989572 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istio-data" (OuterVolumeSpecName: "istio-data") pod "b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429" (UID: "b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:43:22.990846 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.990823 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istio-token" (OuterVolumeSpecName: "istio-token") pod "b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429" (UID: "b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:43:22.990971 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.990954 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429" (UID: "b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:43:22.991036 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.990973 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-kube-api-access-m2tbc" (OuterVolumeSpecName: "kube-api-access-m2tbc") pod "b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429" (UID: "b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429"). InnerVolumeSpecName "kube-api-access-m2tbc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:43:22.991435 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:22.991408 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429" (UID: "b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 13:43:23.090266 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:23.090232 2531 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istio-data\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:43:23.090266 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:23.090266 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2tbc\" (UniqueName: \"kubernetes.io/projected/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-kube-api-access-m2tbc\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:43:23.090517 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:23.090279 2531 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-credential-socket\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:43:23.090517 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:23.090293 2531 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-workload-socket\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:43:23.090517 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:23.090304 2531 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istio-podinfo\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:43:23.090517 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:23.090314 2531 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istio-token\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:43:23.090517 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:23.090325 2531 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429-istio-envoy\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:43:23.254391 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:23.254346 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm"] Jul 23 13:43:23.258586 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:23.258560 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-da798329-openshift-default-857d8c474c-bbnvm"] Jul 23 13:43:23.332432 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:23.332404 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429" path="/var/lib/kubelet/pods/b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429/volumes" Jul 23 13:43:24.294108 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:24.294071 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:43:33.423275 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:33.423198 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:43:43.494798 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:43.494765 2531 scope.go:117] "RemoveContainer" containerID="00ab7ab7c12176bda808c39c7c7cb5d6d707d43987324fa148c7bcdba9ff1081" Jul 23 13:43:43.503541 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:43:43.503509 2531 scope.go:117] "RemoveContainer" containerID="b06b54145e97eba8d3449a7c56d7413afbd7322974a3c3931d00e6a2aae9af0a" Jul 23 13:44:15.213071 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.213041 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc"] Jul 23 13:44:15.213569 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.213431 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429" containerName="istio-proxy" Jul 23 13:44:15.213569 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.213442 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429" containerName="istio-proxy" Jul 23 13:44:15.213569 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.213465 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e84652c-a370-4c77-8c7b-c33800381c7d" containerName="revoke-keys" Jul 23 13:44:15.213569 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.213470 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e84652c-a370-4c77-8c7b-c33800381c7d" containerName="revoke-keys" Jul 23 13:44:15.213569 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.213530 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429" containerName="istio-proxy" Jul 23 13:44:15.213569 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.213537 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e84652c-a370-4c77-8c7b-c33800381c7d" containerName="revoke-keys" Jul 23 13:44:15.213569 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.213545 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429" containerName="istio-proxy" Jul 23 13:44:15.216549 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.216526 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.219037 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.219016 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-98925a-openshift-default-dockercfg-bsvzq\"" Jul 23 13:44:15.235261 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.235235 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc"] Jul 23 13:44:15.262947 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.262918 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-workload-certs\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.263107 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.262960 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istio-token\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.263107 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.263034 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-workload-socket\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.263107 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.263079 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-credential-socket\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.263306 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.263107 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istio-envoy\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.263306 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.263183 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istio-data\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.263306 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.263220 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istio-podinfo\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.263306 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.263247 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istiod-ca-cert\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.263306 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.263287 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqc8d\" (UniqueName: \"kubernetes.io/projected/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-kube-api-access-hqc8d\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.363788 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.363745 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istio-data\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.363999 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.363805 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istio-podinfo\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.363999 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.363832 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istiod-ca-cert\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.363999 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.363867 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqc8d\" (UniqueName: \"kubernetes.io/projected/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-kube-api-access-hqc8d\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.363999 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.363905 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-workload-certs\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.363999 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.363941 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istio-token\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.363999 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.363976 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-workload-socket\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.364306 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.364013 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-credential-socket\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.364306 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.364069 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istio-envoy\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.364306 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.364277 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istio-data\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.364529 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.364475 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-credential-socket\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.364529 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.364507 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-workload-socket\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.364610 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.364544 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-workload-certs\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.364709 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.364686 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istiod-ca-cert\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.366451 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.366432 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istio-envoy\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.366568 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.366550 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istio-podinfo\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.372401 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.372375 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqc8d\" (UniqueName: \"kubernetes.io/projected/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-kube-api-access-hqc8d\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.372725 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.372704 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istio-token\") pod \"e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.526898 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.526805 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:15.668912 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.668887 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc"] Jul 23 13:44:15.670247 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:44:15.670218 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod979d1772_9308_4b41_a7d8_7a4ba4f97e5d.slice/crio-f7ddac92db9fbbf6268028169aca869b1a1f4460c5c4ab0a9ced099cd08fc06b WatchSource:0}: Error finding container f7ddac92db9fbbf6268028169aca869b1a1f4460c5c4ab0a9ced099cd08fc06b: Status 404 returned error can't find the container with id f7ddac92db9fbbf6268028169aca869b1a1f4460c5c4ab0a9ced099cd08fc06b Jul 23 13:44:15.672287 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.672254 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:44:15.672410 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.672317 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:44:15.672410 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:15.672346 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:44:16.242619 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:16.242584 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" event={"ID":"979d1772-9308-4b41-a7d8-7a4ba4f97e5d","Type":"ContainerStarted","Data":"e7a07cbac5978b8de4567d58480603f408ea07b71b75a82f0eda0d7f32add10a"} Jul 23 13:44:16.242982 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:16.242641 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" event={"ID":"979d1772-9308-4b41-a7d8-7a4ba4f97e5d","Type":"ContainerStarted","Data":"f7ddac92db9fbbf6268028169aca869b1a1f4460c5c4ab0a9ced099cd08fc06b"} Jul 23 13:44:16.271392 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:16.271316 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" podStartSLOduration=1.271297121 podStartE2EDuration="1.271297121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:44:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:44:16.268558436 +0000 UTC m=+2193.507834412" watchObservedRunningTime="2026-07-23 13:44:16.271297121 +0000 UTC m=+2193.510573100" Jul 23 13:44:16.527486 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:16.527395 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:16.532286 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:16.532261 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:17.248877 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:17.248845 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:17.249993 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:17.249977 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:23.372915 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.372868 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw"] Jul 23 13:44:23.373503 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.373485 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429" containerName="istio-proxy" Jul 23 13:44:23.373561 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.373507 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4a6d27e-89ac-4174-9c8e-ffa9c9ff8429" containerName="istio-proxy" Jul 23 13:44:23.377324 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.377301 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.380604 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.380580 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-38ab5f-openshift-default-dockercfg-nmsqt\"" Jul 23 13:44:23.391084 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.391058 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw"] Jul 23 13:44:23.434904 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.434867 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istio-data\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.435080 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.434924 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istio-envoy\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.435080 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.434942 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-workload-certs\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.435080 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.434964 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-credential-socket\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.435080 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.434980 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istiod-ca-cert\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.435080 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.435068 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istio-podinfo\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.435299 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.435110 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istio-token\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.435299 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.435131 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8z9n\" (UniqueName: \"kubernetes.io/projected/c6e58e81-aa0e-4fcf-b293-95e0b7852282-kube-api-access-v8z9n\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.435299 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.435156 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-workload-socket\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.540374 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.536075 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istio-envoy\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.540374 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.536159 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-workload-certs\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.540374 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.536206 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-credential-socket\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.540374 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.536241 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istiod-ca-cert\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.540374 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.536347 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istio-podinfo\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.540374 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.536415 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istio-token\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.540374 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.536445 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8z9n\" (UniqueName: \"kubernetes.io/projected/c6e58e81-aa0e-4fcf-b293-95e0b7852282-kube-api-access-v8z9n\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.540374 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.536495 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-workload-socket\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.540374 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.536546 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istio-data\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.540374 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.537035 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-workload-certs\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.540374 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.537335 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-credential-socket\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.540374 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.538044 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istiod-ca-cert\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.541135 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.541020 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istio-podinfo\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.542372 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.541833 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-workload-socket\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.542372 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.542159 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istio-data\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.544426 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.544398 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istio-envoy\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.549989 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.549965 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istio-token\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.550237 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.550214 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8z9n\" (UniqueName: \"kubernetes.io/projected/c6e58e81-aa0e-4fcf-b293-95e0b7852282-kube-api-access-v8z9n\") pod \"e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.688495 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.688400 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:23.690000 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.689973 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:44:23.831695 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.831664 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw"] Jul 23 13:44:23.833570 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:44:23.833543 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6e58e81_aa0e_4fcf_b293_95e0b7852282.slice/crio-f197776d591e8c176f033c9ade185c8db4b6819e8b9058c1841f6ff63c38a870 WatchSource:0}: Error finding container f197776d591e8c176f033c9ade185c8db4b6819e8b9058c1841f6ff63c38a870: Status 404 returned error can't find the container with id f197776d591e8c176f033c9ade185c8db4b6819e8b9058c1841f6ff63c38a870 Jul 23 13:44:23.835658 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.835626 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:44:23.835739 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.835701 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:44:23.835739 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:23.835729 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:44:24.288917 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:24.288882 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" event={"ID":"c6e58e81-aa0e-4fcf-b293-95e0b7852282","Type":"ContainerStarted","Data":"056683a1ac01b740880a9c33012650a58990e3492e41518ef6a1df5cd787c134"} Jul 23 13:44:24.289102 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:24.288935 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" event={"ID":"c6e58e81-aa0e-4fcf-b293-95e0b7852282","Type":"ContainerStarted","Data":"f197776d591e8c176f033c9ade185c8db4b6819e8b9058c1841f6ff63c38a870"} Jul 23 13:44:24.311562 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:24.311511 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" podStartSLOduration=1.311495375 podStartE2EDuration="1.311495375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:44:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:44:24.309511942 +0000 UTC m=+2201.548787916" watchObservedRunningTime="2026-07-23 13:44:24.311495375 +0000 UTC m=+2201.550771350" Jul 23 13:44:24.688982 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:24.688944 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:24.694236 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:24.694212 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:25.295040 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:25.295005 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:25.296099 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:25.296081 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:32.201414 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:32.201373 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:44:32.250576 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:32.250537 2531 patch_prober.go:28] interesting pod/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 23 13:44:32.250745 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:32.250599 2531 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" podUID="979d1772-9308-4b41-a7d8-7a4ba4f97e5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 23 13:44:32.339461 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:32.339429 2531 generic.go:356] "Generic (PLEG): container finished" podID="979d1772-9308-4b41-a7d8-7a4ba4f97e5d" containerID="e7a07cbac5978b8de4567d58480603f408ea07b71b75a82f0eda0d7f32add10a" exitCode=0 Jul 23 13:44:32.339643 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:32.339475 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" event={"ID":"979d1772-9308-4b41-a7d8-7a4ba4f97e5d","Type":"ContainerDied","Data":"e7a07cbac5978b8de4567d58480603f408ea07b71b75a82f0eda0d7f32add10a"} Jul 23 13:44:32.339905 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:32.339881 2531 scope.go:117] "RemoveContainer" containerID="e7a07cbac5978b8de4567d58480603f408ea07b71b75a82f0eda0d7f32add10a" Jul 23 13:44:32.341418 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:32.341380 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:44:32.341546 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:32.341478 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:44:32.341546 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:32.341523 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:44:33.353006 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:33.352956 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" event={"ID":"979d1772-9308-4b41-a7d8-7a4ba4f97e5d","Type":"ContainerStarted","Data":"9f22f370647bbc368460559ad74ccf9b7557d86a1e2550dfe0ccd80f2e57328e"} Jul 23 13:44:33.528130 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:33.528093 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:33.528306 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:33.528200 2531 patch_prober.go:28] interesting pod/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 23 13:44:33.528306 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:33.528232 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" podUID="979d1772-9308-4b41-a7d8-7a4ba4f97e5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 23 13:44:34.527212 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:34.527172 2531 patch_prober.go:28] interesting pod/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 23 13:44:34.527640 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:34.527242 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" podUID="979d1772-9308-4b41-a7d8-7a4ba4f97e5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 23 13:44:35.527518 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:35.527482 2531 patch_prober.go:28] interesting pod/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 23 13:44:35.527898 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:35.527537 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" podUID="979d1772-9308-4b41-a7d8-7a4ba4f97e5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 23 13:44:36.528101 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:36.528061 2531 patch_prober.go:28] interesting pod/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 23 13:44:36.528547 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:36.528130 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" podUID="979d1772-9308-4b41-a7d8-7a4ba4f97e5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 23 13:44:37.531567 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:37.531540 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:37.531920 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:37.531831 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:37.532719 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:37.532703 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:44:40.346872 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:40.346841 2531 patch_prober.go:28] interesting pod/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": read tcp 10.134.0.2:58694->10.134.0.59:15021: read: connection reset by peer" start-of-body= Jul 23 13:44:40.347504 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:40.346918 2531 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" podUID="c6e58e81-aa0e-4fcf-b293-95e0b7852282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": read tcp 10.134.0.2:58694->10.134.0.59:15021: read: connection reset by peer" Jul 23 13:44:40.407211 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:40.407182 2531 generic.go:356] "Generic (PLEG): container finished" podID="c6e58e81-aa0e-4fcf-b293-95e0b7852282" containerID="056683a1ac01b740880a9c33012650a58990e3492e41518ef6a1df5cd787c134" exitCode=0 Jul 23 13:44:40.407389 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:40.407259 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" event={"ID":"c6e58e81-aa0e-4fcf-b293-95e0b7852282","Type":"ContainerDied","Data":"056683a1ac01b740880a9c33012650a58990e3492e41518ef6a1df5cd787c134"} Jul 23 13:44:40.407702 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:40.407688 2531 scope.go:117] "RemoveContainer" containerID="056683a1ac01b740880a9c33012650a58990e3492e41518ef6a1df5cd787c134" Jul 23 13:44:40.408738 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:40.408709 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:44:40.408805 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:40.408791 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:44:40.408842 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:40.408827 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:44:41.415342 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:41.415307 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" event={"ID":"c6e58e81-aa0e-4fcf-b293-95e0b7852282","Type":"ContainerStarted","Data":"8119a8b5717522e41d3f8e3703c5af009d52aa743f491643937b159f6061de2a"} Jul 23 13:44:41.688808 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:41.688709 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:41.688987 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:41.688963 2531 patch_prober.go:28] interesting pod/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 23 13:44:41.689034 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:41.689010 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" podUID="c6e58e81-aa0e-4fcf-b293-95e0b7852282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 23 13:44:42.689213 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:42.689183 2531 patch_prober.go:28] interesting pod/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 23 13:44:42.689577 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:42.689238 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" podUID="c6e58e81-aa0e-4fcf-b293-95e0b7852282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 23 13:44:43.534932 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:43.534890 2531 scope.go:117] "RemoveContainer" containerID="bffdebb999fa1ffc7f59498d77c9325fdf8c085c9a5b39a41285048b63d40fea" Jul 23 13:44:43.689517 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:43.689489 2531 patch_prober.go:28] interesting pod/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 23 13:44:43.689876 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:43.689548 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" podUID="c6e58e81-aa0e-4fcf-b293-95e0b7852282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 23 13:44:44.689758 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:44.689718 2531 patch_prober.go:28] interesting pod/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 23 13:44:44.690143 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:44.689787 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" podUID="c6e58e81-aa0e-4fcf-b293-95e0b7852282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 23 13:44:45.396773 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:45.396735 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:44:45.692093 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:45.692011 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:45.692510 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:45.692292 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:45.693003 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:45.692985 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:44:59.391506 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:44:59.391470 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:45:00.140390 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:45:00.140335 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-38ab5f-29746905-z5ntt"] Jul 23 13:45:00.143219 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:45:00.143196 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-38ab5f-29746905-z5ntt" Jul 23 13:45:00.146965 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:45:00.146943 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vp6nn\"" Jul 23 13:45:00.153454 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:45:00.153432 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-38ab5f-29746905-z5ntt" Jul 23 13:45:00.159229 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:45:00.159200 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-38ab5f-29746905-z5ntt"] Jul 23 13:45:00.295503 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:45:00.295480 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-38ab5f-29746905-z5ntt"] Jul 23 13:45:00.297209 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:45:00.297184 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb15e5a1c_1f8c_48f7_afc2_6e132840389e.slice/crio-ff21e7c619e0218d7b563446a43389776e1125483e976569d23042cbc1012f51 WatchSource:0}: Error finding container ff21e7c619e0218d7b563446a43389776e1125483e976569d23042cbc1012f51: Status 404 returned error can't find the container with id ff21e7c619e0218d7b563446a43389776e1125483e976569d23042cbc1012f51 Jul 23 13:45:00.523998 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:45:00.523965 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-38ab5f-29746905-z5ntt" event={"ID":"b15e5a1c-1f8c-48f7-afc2-6e132840389e","Type":"ContainerStarted","Data":"ff21e7c619e0218d7b563446a43389776e1125483e976569d23042cbc1012f51"} Jul 23 13:45:01.531211 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:45:01.531119 2531 generic.go:356] "Generic (PLEG): container finished" podID="b15e5a1c-1f8c-48f7-afc2-6e132840389e" containerID="24854a0c083f803f72a3da321441cfcbec27b76fecca5a25d05cb5bd40de488b" exitCode=0 Jul 23 13:45:01.531601 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:45:01.531210 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-38ab5f-29746905-z5ntt" event={"ID":"b15e5a1c-1f8c-48f7-afc2-6e132840389e","Type":"ContainerDied","Data":"24854a0c083f803f72a3da321441cfcbec27b76fecca5a25d05cb5bd40de488b"} Jul 23 13:45:02.677376 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:45:02.677334 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-38ab5f-29746905-z5ntt" Jul 23 13:45:03.543713 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:45:03.543648 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-38ab5f-29746905-z5ntt" Jul 23 13:45:03.543867 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:45:03.543645 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-38ab5f-29746905-z5ntt" event={"ID":"b15e5a1c-1f8c-48f7-afc2-6e132840389e","Type":"ContainerDied","Data":"ff21e7c619e0218d7b563446a43389776e1125483e976569d23042cbc1012f51"} Jul 23 13:45:03.543867 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:45:03.543768 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff21e7c619e0218d7b563446a43389776e1125483e976569d23042cbc1012f51" Jul 23 13:45:19.997855 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:45:19.997818 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:45:33.508159 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:45:33.508115 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:46:08.088081 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:08.088035 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:46:17.699068 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:17.699037 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:46:24.014410 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:24.014378 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc"] Jul 23 13:46:24.014893 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:24.014596 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" podUID="979d1772-9308-4b41-a7d8-7a4ba4f97e5d" containerName="istio-proxy" containerID="cri-o://9f22f370647bbc368460559ad74ccf9b7557d86a1e2550dfe0ccd80f2e57328e" gracePeriod=30 Jul 23 13:46:29.269655 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.269621 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:46:29.394973 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.394893 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istiod-ca-cert\") pod \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " Jul 23 13:46:29.394973 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.394968 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-credential-socket\") pod \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " Jul 23 13:46:29.395170 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.395000 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istio-token\") pod \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " Jul 23 13:46:29.395170 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.395028 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-workload-certs\") pod \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " Jul 23 13:46:29.395170 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.395062 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-workload-socket\") pod \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " Jul 23 13:46:29.395170 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.395093 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istio-envoy\") pod \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " Jul 23 13:46:29.395170 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.395141 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istio-podinfo\") pod \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " Jul 23 13:46:29.395451 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.395258 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istio-data\") pod \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " Jul 23 13:46:29.395451 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.395288 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqc8d\" (UniqueName: \"kubernetes.io/projected/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-kube-api-access-hqc8d\") pod \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\" (UID: \"979d1772-9308-4b41-a7d8-7a4ba4f97e5d\") " Jul 23 13:46:29.395451 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.395308 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "979d1772-9308-4b41-a7d8-7a4ba4f97e5d" (UID: "979d1772-9308-4b41-a7d8-7a4ba4f97e5d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:46:29.395451 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.395428 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "979d1772-9308-4b41-a7d8-7a4ba4f97e5d" (UID: "979d1772-9308-4b41-a7d8-7a4ba4f97e5d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:46:29.395451 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.395316 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "979d1772-9308-4b41-a7d8-7a4ba4f97e5d" (UID: "979d1772-9308-4b41-a7d8-7a4ba4f97e5d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:46:29.395693 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.395591 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "979d1772-9308-4b41-a7d8-7a4ba4f97e5d" (UID: "979d1772-9308-4b41-a7d8-7a4ba4f97e5d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:46:29.395793 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.395768 2531 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istiod-ca-cert\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:46:29.395847 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.395793 2531 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-credential-socket\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:46:29.395847 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.395807 2531 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-workload-certs\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:46:29.395847 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.395821 2531 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-workload-socket\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:46:29.396582 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.396555 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istio-data" (OuterVolumeSpecName: "istio-data") pod "979d1772-9308-4b41-a7d8-7a4ba4f97e5d" (UID: "979d1772-9308-4b41-a7d8-7a4ba4f97e5d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:46:29.397525 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.397494 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "979d1772-9308-4b41-a7d8-7a4ba4f97e5d" (UID: "979d1772-9308-4b41-a7d8-7a4ba4f97e5d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:46:29.397585 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.397526 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istio-token" (OuterVolumeSpecName: "istio-token") pod "979d1772-9308-4b41-a7d8-7a4ba4f97e5d" (UID: "979d1772-9308-4b41-a7d8-7a4ba4f97e5d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:46:29.397939 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.397905 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "979d1772-9308-4b41-a7d8-7a4ba4f97e5d" (UID: "979d1772-9308-4b41-a7d8-7a4ba4f97e5d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 13:46:29.398002 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.397932 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-kube-api-access-hqc8d" (OuterVolumeSpecName: "kube-api-access-hqc8d") pod "979d1772-9308-4b41-a7d8-7a4ba4f97e5d" (UID: "979d1772-9308-4b41-a7d8-7a4ba4f97e5d"). InnerVolumeSpecName "kube-api-access-hqc8d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:46:29.498584 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.497646 2531 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istio-podinfo\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:46:29.498584 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.497687 2531 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istio-data\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:46:29.498584 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.497710 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqc8d\" (UniqueName: \"kubernetes.io/projected/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-kube-api-access-hqc8d\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:46:29.498584 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.497724 2531 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istio-token\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:46:29.498584 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:29.497738 2531 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/979d1772-9308-4b41-a7d8-7a4ba4f97e5d-istio-envoy\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:46:30.039121 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:30.039088 2531 generic.go:356] "Generic (PLEG): container finished" podID="979d1772-9308-4b41-a7d8-7a4ba4f97e5d" containerID="9f22f370647bbc368460559ad74ccf9b7557d86a1e2550dfe0ccd80f2e57328e" exitCode=0 Jul 23 13:46:30.039301 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:30.039140 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" event={"ID":"979d1772-9308-4b41-a7d8-7a4ba4f97e5d","Type":"ContainerDied","Data":"9f22f370647bbc368460559ad74ccf9b7557d86a1e2550dfe0ccd80f2e57328e"} Jul 23 13:46:30.039301 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:30.039162 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" Jul 23 13:46:30.039301 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:30.039225 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc" event={"ID":"979d1772-9308-4b41-a7d8-7a4ba4f97e5d","Type":"ContainerDied","Data":"f7ddac92db9fbbf6268028169aca869b1a1f4460c5c4ab0a9ced099cd08fc06b"} Jul 23 13:46:30.039301 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:30.039258 2531 scope.go:117] "RemoveContainer" containerID="9f22f370647bbc368460559ad74ccf9b7557d86a1e2550dfe0ccd80f2e57328e" Jul 23 13:46:30.049299 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:30.049278 2531 scope.go:117] "RemoveContainer" containerID="e7a07cbac5978b8de4567d58480603f408ea07b71b75a82f0eda0d7f32add10a" Jul 23 13:46:30.061207 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:30.061179 2531 scope.go:117] "RemoveContainer" containerID="9f22f370647bbc368460559ad74ccf9b7557d86a1e2550dfe0ccd80f2e57328e" Jul 23 13:46:30.061711 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:46:30.061681 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f22f370647bbc368460559ad74ccf9b7557d86a1e2550dfe0ccd80f2e57328e\": container with ID starting with 9f22f370647bbc368460559ad74ccf9b7557d86a1e2550dfe0ccd80f2e57328e not found: ID does not exist" containerID="9f22f370647bbc368460559ad74ccf9b7557d86a1e2550dfe0ccd80f2e57328e" Jul 23 13:46:30.061828 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:30.061721 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f22f370647bbc368460559ad74ccf9b7557d86a1e2550dfe0ccd80f2e57328e"} err="failed to get container status \"9f22f370647bbc368460559ad74ccf9b7557d86a1e2550dfe0ccd80f2e57328e\": rpc error: code = NotFound desc = could not find container \"9f22f370647bbc368460559ad74ccf9b7557d86a1e2550dfe0ccd80f2e57328e\": container with ID starting with 9f22f370647bbc368460559ad74ccf9b7557d86a1e2550dfe0ccd80f2e57328e not found: ID does not exist" Jul 23 13:46:30.061828 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:30.061746 2531 scope.go:117] "RemoveContainer" containerID="e7a07cbac5978b8de4567d58480603f408ea07b71b75a82f0eda0d7f32add10a" Jul 23 13:46:30.062065 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:46:30.062039 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7a07cbac5978b8de4567d58480603f408ea07b71b75a82f0eda0d7f32add10a\": container with ID starting with e7a07cbac5978b8de4567d58480603f408ea07b71b75a82f0eda0d7f32add10a not found: ID does not exist" containerID="e7a07cbac5978b8de4567d58480603f408ea07b71b75a82f0eda0d7f32add10a" Jul 23 13:46:30.062108 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:30.062075 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7a07cbac5978b8de4567d58480603f408ea07b71b75a82f0eda0d7f32add10a"} err="failed to get container status \"e7a07cbac5978b8de4567d58480603f408ea07b71b75a82f0eda0d7f32add10a\": rpc error: code = NotFound desc = could not find container \"e7a07cbac5978b8de4567d58480603f408ea07b71b75a82f0eda0d7f32add10a\": container with ID starting with e7a07cbac5978b8de4567d58480603f408ea07b71b75a82f0eda0d7f32add10a not found: ID does not exist" Jul 23 13:46:30.064535 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:30.064511 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc"] Jul 23 13:46:30.066858 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:30.066836 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-98925a-openshift-default-7db68b785c-kn2tc"] Jul 23 13:46:31.333795 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:31.333759 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="979d1772-9308-4b41-a7d8-7a4ba4f97e5d" path="/var/lib/kubelet/pods/979d1772-9308-4b41-a7d8-7a4ba4f97e5d/volumes" Jul 23 13:46:34.895920 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:34.895836 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-38ab5f-29746905-z5ntt"] Jul 23 13:46:34.899103 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:34.899078 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-38ab5f-29746905-z5ntt"] Jul 23 13:46:35.332937 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:35.332903 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b15e5a1c-1f8c-48f7-afc2-6e132840389e" path="/var/lib/kubelet/pods/b15e5a1c-1f8c-48f7-afc2-6e132840389e/volumes" Jul 23 13:46:37.682033 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:37.681995 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:46:43.579345 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:43.579305 2531 scope.go:117] "RemoveContainer" containerID="d8566dfca112f7e020d3411a0563585d3ba99788e59df697487db31909643469" Jul 23 13:46:46.138366 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:46.138329 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw"] Jul 23 13:46:46.138753 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:46.138558 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" podUID="c6e58e81-aa0e-4fcf-b293-95e0b7852282" containerName="istio-proxy" containerID="cri-o://8119a8b5717522e41d3f8e3703c5af009d52aa743f491643937b159f6061de2a" gracePeriod=30 Jul 23 13:46:47.130762 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.130719 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr"] Jul 23 13:46:47.131247 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.131230 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="979d1772-9308-4b41-a7d8-7a4ba4f97e5d" containerName="istio-proxy" Jul 23 13:46:47.131247 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.131248 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="979d1772-9308-4b41-a7d8-7a4ba4f97e5d" containerName="istio-proxy" Jul 23 13:46:47.131393 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.131271 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="979d1772-9308-4b41-a7d8-7a4ba4f97e5d" containerName="istio-proxy" Jul 23 13:46:47.131393 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.131281 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="979d1772-9308-4b41-a7d8-7a4ba4f97e5d" containerName="istio-proxy" Jul 23 13:46:47.131393 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.131297 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b15e5a1c-1f8c-48f7-afc2-6e132840389e" containerName="cleanup" Jul 23 13:46:47.131393 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.131302 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="b15e5a1c-1f8c-48f7-afc2-6e132840389e" containerName="cleanup" Jul 23 13:46:47.131523 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.131401 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="979d1772-9308-4b41-a7d8-7a4ba4f97e5d" containerName="istio-proxy" Jul 23 13:46:47.131523 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.131410 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="b15e5a1c-1f8c-48f7-afc2-6e132840389e" containerName="cleanup" Jul 23 13:46:47.134204 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.134182 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.137830 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.137810 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-636f5e-openshift-default-dockercfg-khsbm\"" Jul 23 13:46:47.159154 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.159130 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istio-token\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.159603 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.159173 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-workload-certs\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.159603 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.159196 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwlpl\" (UniqueName: \"kubernetes.io/projected/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-kube-api-access-wwlpl\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.159603 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.159236 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istiod-ca-cert\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.159603 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.159257 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istio-data\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.159603 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.159330 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istio-envoy\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.159603 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.159418 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istio-podinfo\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.159603 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.159487 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-workload-socket\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.159603 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.159522 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-credential-socket\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.159890 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.159786 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr"] Jul 23 13:46:47.260151 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.260111 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-workload-socket\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.260151 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.260150 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-credential-socket\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.260425 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.260177 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istio-token\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.260425 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.260212 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-workload-certs\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.260425 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.260234 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwlpl\" (UniqueName: \"kubernetes.io/projected/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-kube-api-access-wwlpl\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.260425 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.260261 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istiod-ca-cert\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.260425 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.260290 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istio-data\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.260425 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.260328 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istio-envoy\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.260763 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.260542 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istio-podinfo\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.260763 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.260666 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-credential-socket\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.260763 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.260701 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-workload-socket\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.260928 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.260834 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istio-data\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.260928 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.260867 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-workload-certs\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.261098 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.261078 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istiod-ca-cert\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.262861 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.262841 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istio-envoy\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.263088 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.263071 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istio-podinfo\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.270067 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.270020 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istio-token\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.270404 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.270384 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwlpl\" (UniqueName: \"kubernetes.io/projected/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-kube-api-access-wwlpl\") pod \"e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.444326 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.444224 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:47.591670 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.591642 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr"] Jul 23 13:46:47.593417 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:46:47.593391 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89ffc6c4_0f45_4ac4_aa19_8442c77d0eb6.slice/crio-050741c355a0b33bb21a6abdf7b6b67e7b18114117dc5ba7a39325a75814534c WatchSource:0}: Error finding container 050741c355a0b33bb21a6abdf7b6b67e7b18114117dc5ba7a39325a75814534c: Status 404 returned error can't find the container with id 050741c355a0b33bb21a6abdf7b6b67e7b18114117dc5ba7a39325a75814534c Jul 23 13:46:47.596283 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.596241 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:46:47.596416 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.596334 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:46:47.596416 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:47.596407 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:46:48.142772 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:48.142740 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" event={"ID":"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6","Type":"ContainerStarted","Data":"338379ec5f2476d75a5df2549e4901f1f7ed1094801f4590fd3d5b39ff0a3815"} Jul 23 13:46:48.142953 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:48.142791 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" event={"ID":"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6","Type":"ContainerStarted","Data":"050741c355a0b33bb21a6abdf7b6b67e7b18114117dc5ba7a39325a75814534c"} Jul 23 13:46:48.164613 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:48.164564 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" podStartSLOduration=1.164548023 podStartE2EDuration="1.164548023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:46:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:46:48.162866412 +0000 UTC m=+2345.402142401" watchObservedRunningTime="2026-07-23 13:46:48.164548023 +0000 UTC m=+2345.403823998" Jul 23 13:46:48.444860 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:48.444772 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:48.449763 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:48.449734 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:49.148246 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:49.148216 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:49.149099 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:49.149081 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:46:51.392422 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.392398 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:46:51.496859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.496768 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-workload-socket\") pod \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " Jul 23 13:46:51.496859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.496815 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-workload-certs\") pod \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " Jul 23 13:46:51.496859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.496845 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istiod-ca-cert\") pod \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " Jul 23 13:46:51.497155 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.496872 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-credential-socket\") pod \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " Jul 23 13:46:51.497155 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.496909 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istio-data\") pod \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " Jul 23 13:46:51.497155 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.496931 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istio-envoy\") pod \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " Jul 23 13:46:51.497155 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.496982 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istio-token\") pod \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " Jul 23 13:46:51.497155 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.497032 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8z9n\" (UniqueName: \"kubernetes.io/projected/c6e58e81-aa0e-4fcf-b293-95e0b7852282-kube-api-access-v8z9n\") pod \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " Jul 23 13:46:51.497155 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.497064 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istio-podinfo\") pod \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\" (UID: \"c6e58e81-aa0e-4fcf-b293-95e0b7852282\") " Jul 23 13:46:51.497155 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.497116 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c6e58e81-aa0e-4fcf-b293-95e0b7852282" (UID: "c6e58e81-aa0e-4fcf-b293-95e0b7852282"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:46:51.497551 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.497180 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c6e58e81-aa0e-4fcf-b293-95e0b7852282" (UID: "c6e58e81-aa0e-4fcf-b293-95e0b7852282"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:46:51.497551 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.497382 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c6e58e81-aa0e-4fcf-b293-95e0b7852282" (UID: "c6e58e81-aa0e-4fcf-b293-95e0b7852282"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:46:51.497551 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.497436 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c6e58e81-aa0e-4fcf-b293-95e0b7852282" (UID: "c6e58e81-aa0e-4fcf-b293-95e0b7852282"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:46:51.497551 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.497484 2531 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-workload-socket\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:46:51.497551 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.497502 2531 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-workload-certs\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:46:51.497551 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.497517 2531 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-credential-socket\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:46:51.497990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.497971 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istio-data" (OuterVolumeSpecName: "istio-data") pod "c6e58e81-aa0e-4fcf-b293-95e0b7852282" (UID: "c6e58e81-aa0e-4fcf-b293-95e0b7852282"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:46:51.499794 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.499765 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istio-token" (OuterVolumeSpecName: "istio-token") pod "c6e58e81-aa0e-4fcf-b293-95e0b7852282" (UID: "c6e58e81-aa0e-4fcf-b293-95e0b7852282"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:46:51.499907 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.499792 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c6e58e81-aa0e-4fcf-b293-95e0b7852282" (UID: "c6e58e81-aa0e-4fcf-b293-95e0b7852282"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:46:51.499907 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.499883 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6e58e81-aa0e-4fcf-b293-95e0b7852282-kube-api-access-v8z9n" (OuterVolumeSpecName: "kube-api-access-v8z9n") pod "c6e58e81-aa0e-4fcf-b293-95e0b7852282" (UID: "c6e58e81-aa0e-4fcf-b293-95e0b7852282"). InnerVolumeSpecName "kube-api-access-v8z9n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:46:51.499907 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.499895 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c6e58e81-aa0e-4fcf-b293-95e0b7852282" (UID: "c6e58e81-aa0e-4fcf-b293-95e0b7852282"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 13:46:51.604795 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.599534 2531 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istiod-ca-cert\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:46:51.604795 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.599577 2531 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istio-data\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:46:51.604795 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.599592 2531 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istio-envoy\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:46:51.604795 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.599604 2531 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istio-token\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:46:51.604795 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.599617 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8z9n\" (UniqueName: \"kubernetes.io/projected/c6e58e81-aa0e-4fcf-b293-95e0b7852282-kube-api-access-v8z9n\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:46:51.604795 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:51.599636 2531 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6e58e81-aa0e-4fcf-b293-95e0b7852282-istio-podinfo\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:46:52.167276 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:52.167246 2531 generic.go:356] "Generic (PLEG): container finished" podID="c6e58e81-aa0e-4fcf-b293-95e0b7852282" containerID="8119a8b5717522e41d3f8e3703c5af009d52aa743f491643937b159f6061de2a" exitCode=0 Jul 23 13:46:52.167512 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:52.167317 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" Jul 23 13:46:52.167512 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:52.167318 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" event={"ID":"c6e58e81-aa0e-4fcf-b293-95e0b7852282","Type":"ContainerDied","Data":"8119a8b5717522e41d3f8e3703c5af009d52aa743f491643937b159f6061de2a"} Jul 23 13:46:52.167512 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:52.167453 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw" event={"ID":"c6e58e81-aa0e-4fcf-b293-95e0b7852282","Type":"ContainerDied","Data":"f197776d591e8c176f033c9ade185c8db4b6819e8b9058c1841f6ff63c38a870"} Jul 23 13:46:52.167512 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:52.167477 2531 scope.go:117] "RemoveContainer" containerID="8119a8b5717522e41d3f8e3703c5af009d52aa743f491643937b159f6061de2a" Jul 23 13:46:52.177236 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:52.177219 2531 scope.go:117] "RemoveContainer" containerID="056683a1ac01b740880a9c33012650a58990e3492e41518ef6a1df5cd787c134" Jul 23 13:46:52.185427 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:52.185407 2531 scope.go:117] "RemoveContainer" containerID="8119a8b5717522e41d3f8e3703c5af009d52aa743f491643937b159f6061de2a" Jul 23 13:46:52.185677 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:46:52.185659 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8119a8b5717522e41d3f8e3703c5af009d52aa743f491643937b159f6061de2a\": container with ID starting with 8119a8b5717522e41d3f8e3703c5af009d52aa743f491643937b159f6061de2a not found: ID does not exist" containerID="8119a8b5717522e41d3f8e3703c5af009d52aa743f491643937b159f6061de2a" Jul 23 13:46:52.185737 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:52.185684 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8119a8b5717522e41d3f8e3703c5af009d52aa743f491643937b159f6061de2a"} err="failed to get container status \"8119a8b5717522e41d3f8e3703c5af009d52aa743f491643937b159f6061de2a\": rpc error: code = NotFound desc = could not find container \"8119a8b5717522e41d3f8e3703c5af009d52aa743f491643937b159f6061de2a\": container with ID starting with 8119a8b5717522e41d3f8e3703c5af009d52aa743f491643937b159f6061de2a not found: ID does not exist" Jul 23 13:46:52.185737 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:52.185700 2531 scope.go:117] "RemoveContainer" containerID="056683a1ac01b740880a9c33012650a58990e3492e41518ef6a1df5cd787c134" Jul 23 13:46:52.185910 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:46:52.185895 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"056683a1ac01b740880a9c33012650a58990e3492e41518ef6a1df5cd787c134\": container with ID starting with 056683a1ac01b740880a9c33012650a58990e3492e41518ef6a1df5cd787c134 not found: ID does not exist" containerID="056683a1ac01b740880a9c33012650a58990e3492e41518ef6a1df5cd787c134" Jul 23 13:46:52.185949 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:52.185914 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"056683a1ac01b740880a9c33012650a58990e3492e41518ef6a1df5cd787c134"} err="failed to get container status \"056683a1ac01b740880a9c33012650a58990e3492e41518ef6a1df5cd787c134\": rpc error: code = NotFound desc = could not find container \"056683a1ac01b740880a9c33012650a58990e3492e41518ef6a1df5cd787c134\": container with ID starting with 056683a1ac01b740880a9c33012650a58990e3492e41518ef6a1df5cd787c134 not found: ID does not exist" Jul 23 13:46:52.203568 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:52.203543 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw"] Jul 23 13:46:52.217975 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:52.217953 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-38ab5f-openshift-default-7bb675d965-trgkw"] Jul 23 13:46:53.333446 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:53.333416 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6e58e81-aa0e-4fcf-b293-95e0b7852282" path="/var/lib/kubelet/pods/c6e58e81-aa0e-4fcf-b293-95e0b7852282/volumes" Jul 23 13:46:55.444613 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.444585 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh"] Jul 23 13:46:55.445011 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.444996 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6e58e81-aa0e-4fcf-b293-95e0b7852282" containerName="istio-proxy" Jul 23 13:46:55.445058 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.445013 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6e58e81-aa0e-4fcf-b293-95e0b7852282" containerName="istio-proxy" Jul 23 13:46:55.445058 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.445032 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6e58e81-aa0e-4fcf-b293-95e0b7852282" containerName="istio-proxy" Jul 23 13:46:55.445058 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.445037 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6e58e81-aa0e-4fcf-b293-95e0b7852282" containerName="istio-proxy" Jul 23 13:46:55.445150 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.445098 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="979d1772-9308-4b41-a7d8-7a4ba4f97e5d" containerName="istio-proxy" Jul 23 13:46:55.445150 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.445107 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6e58e81-aa0e-4fcf-b293-95e0b7852282" containerName="istio-proxy" Jul 23 13:46:55.445150 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.445115 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6e58e81-aa0e-4fcf-b293-95e0b7852282" containerName="istio-proxy" Jul 23 13:46:55.449728 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.449708 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.452734 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.452716 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-a6a223-openshift-default-dockercfg-55qh7\"" Jul 23 13:46:55.462880 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.462857 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh"] Jul 23 13:46:55.535175 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.535132 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istiod-ca-cert\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.535395 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.535228 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istio-data\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.535395 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.535262 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istio-podinfo\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.535395 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.535292 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istio-envoy\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.535591 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.535408 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-workload-socket\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.535591 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.535457 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-workload-certs\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.535591 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.535491 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjlc9\" (UniqueName: \"kubernetes.io/projected/27051ec8-d33a-4e98-90a2-fe07abf01b5f-kube-api-access-vjlc9\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.535591 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.535560 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-credential-socket\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.535841 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.535599 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istio-token\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.636286 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.636253 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istio-envoy\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.636517 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.636302 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-workload-socket\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.636517 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.636324 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-workload-certs\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.636517 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.636452 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjlc9\" (UniqueName: \"kubernetes.io/projected/27051ec8-d33a-4e98-90a2-fe07abf01b5f-kube-api-access-vjlc9\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.636689 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.636521 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-credential-socket\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.636689 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.636552 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istio-token\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.636689 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.636597 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istiod-ca-cert\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.636689 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.636655 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istio-data\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.636689 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.636685 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istio-podinfo\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.636929 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.636798 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-workload-socket\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.636929 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.636796 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-workload-certs\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.636929 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.636875 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-credential-socket\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.637090 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.637013 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istio-data\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.637394 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.637346 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istiod-ca-cert\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.638668 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.638648 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istio-envoy\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.639005 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.638989 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istio-podinfo\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.644656 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.644635 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istio-token\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.644990 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.644973 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjlc9\" (UniqueName: \"kubernetes.io/projected/27051ec8-d33a-4e98-90a2-fe07abf01b5f-kube-api-access-vjlc9\") pod \"e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.759579 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.759496 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:55.905047 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.905017 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh"] Jul 23 13:46:55.906492 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:46:55.906466 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27051ec8_d33a_4e98_90a2_fe07abf01b5f.slice/crio-cc40ba5014f26bf787ad7b2818fc335caa58f68bdb7458aa7ed7bb687ac54f73 WatchSource:0}: Error finding container cc40ba5014f26bf787ad7b2818fc335caa58f68bdb7458aa7ed7bb687ac54f73: Status 404 returned error can't find the container with id cc40ba5014f26bf787ad7b2818fc335caa58f68bdb7458aa7ed7bb687ac54f73 Jul 23 13:46:55.908633 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.908602 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:46:55.908707 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.908667 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:46:55.908707 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:55.908696 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:46:56.195582 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:56.195547 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" event={"ID":"27051ec8-d33a-4e98-90a2-fe07abf01b5f","Type":"ContainerStarted","Data":"778f8fc3a2b89a0a3a9e4b8a5ef25382da871f1ce64daee726827e2733bfc724"} Jul 23 13:46:56.195738 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:56.195627 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" event={"ID":"27051ec8-d33a-4e98-90a2-fe07abf01b5f","Type":"ContainerStarted","Data":"cc40ba5014f26bf787ad7b2818fc335caa58f68bdb7458aa7ed7bb687ac54f73"} Jul 23 13:46:56.229737 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:56.229684 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" podStartSLOduration=1.229668188 podStartE2EDuration="1.229668188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:46:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:46:56.226712959 +0000 UTC m=+2353.465988936" watchObservedRunningTime="2026-07-23 13:46:56.229668188 +0000 UTC m=+2353.468944164" Jul 23 13:46:56.759710 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:56.759668 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:56.764553 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:56.764527 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:56.893797 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:56.893766 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:46:57.201503 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:57.201466 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:46:57.202451 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:46:57.202426 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:47:05.190532 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:05.190488 2531 patch_prober.go:28] interesting pod/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 23 13:47:05.190985 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:05.190559 2531 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" podUID="89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 23 13:47:06.262495 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:06.262463 2531 generic.go:356] "Generic (PLEG): container finished" podID="89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" containerID="338379ec5f2476d75a5df2549e4901f1f7ed1094801f4590fd3d5b39ff0a3815" exitCode=0 Jul 23 13:47:06.262920 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:06.262539 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" event={"ID":"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6","Type":"ContainerDied","Data":"338379ec5f2476d75a5df2549e4901f1f7ed1094801f4590fd3d5b39ff0a3815"} Jul 23 13:47:06.262964 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:06.262927 2531 scope.go:117] "RemoveContainer" containerID="338379ec5f2476d75a5df2549e4901f1f7ed1094801f4590fd3d5b39ff0a3815" Jul 23 13:47:06.263874 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:06.263843 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:47:06.263986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:06.263943 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:47:06.264047 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:06.263995 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:47:07.270446 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:07.270414 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" event={"ID":"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6","Type":"ContainerStarted","Data":"8ac390d5d0501c1db79b4875c8e5bcbe838b23d223f15fdfb80afbb42836e662"} Jul 23 13:47:07.444873 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:07.444836 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:47:07.445441 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:07.445101 2531 patch_prober.go:28] interesting pod/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 23 13:47:07.445441 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:07.445155 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" podUID="89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 23 13:47:08.318205 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:08.318170 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:47:08.444896 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:08.444848 2531 patch_prober.go:28] interesting pod/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 23 13:47:08.445106 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:08.444947 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" podUID="89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 23 13:47:09.445429 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:09.445386 2531 patch_prober.go:28] interesting pod/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 23 13:47:09.445931 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:09.445467 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" podUID="89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 23 13:47:10.444936 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:10.444894 2531 patch_prober.go:28] interesting pod/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 23 13:47:10.445231 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:10.445198 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" podUID="89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 23 13:47:11.445388 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:11.444914 2531 patch_prober.go:28] interesting pod/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 23 13:47:11.445388 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:11.444979 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" podUID="89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 23 13:47:12.203551 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:12.203515 2531 patch_prober.go:28] interesting pod/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 23 13:47:12.203733 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:12.203569 2531 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" podUID="27051ec8-d33a-4e98-90a2-fe07abf01b5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 23 13:47:12.306549 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:12.306514 2531 generic.go:356] "Generic (PLEG): container finished" podID="27051ec8-d33a-4e98-90a2-fe07abf01b5f" containerID="778f8fc3a2b89a0a3a9e4b8a5ef25382da871f1ce64daee726827e2733bfc724" exitCode=0 Jul 23 13:47:12.306713 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:12.306575 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" event={"ID":"27051ec8-d33a-4e98-90a2-fe07abf01b5f","Type":"ContainerDied","Data":"778f8fc3a2b89a0a3a9e4b8a5ef25382da871f1ce64daee726827e2733bfc724"} Jul 23 13:47:12.306973 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:12.306960 2531 scope.go:117] "RemoveContainer" containerID="778f8fc3a2b89a0a3a9e4b8a5ef25382da871f1ce64daee726827e2733bfc724" Jul 23 13:47:12.308061 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:12.308032 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:47:12.308117 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:12.308102 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:47:12.308153 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:12.308136 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:47:12.448544 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:12.448491 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:47:12.449008 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:12.448865 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:47:12.449659 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:12.449636 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:47:13.323975 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:13.323935 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" event={"ID":"27051ec8-d33a-4e98-90a2-fe07abf01b5f","Type":"ContainerStarted","Data":"4e0827be1046d700eb1b9a23aebed83d9b410bf0706640c8f5fbc3d8474449eb"} Jul 23 13:47:13.760109 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:13.759996 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:47:13.760641 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:13.760198 2531 patch_prober.go:28] interesting pod/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 23 13:47:13.760641 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:13.760263 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" podUID="27051ec8-d33a-4e98-90a2-fe07abf01b5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 23 13:47:14.760254 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:14.760215 2531 patch_prober.go:28] interesting pod/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 23 13:47:14.760775 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:14.760275 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" podUID="27051ec8-d33a-4e98-90a2-fe07abf01b5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 23 13:47:15.760338 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:15.760302 2531 patch_prober.go:28] interesting pod/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 23 13:47:15.760750 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:15.760389 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" podUID="27051ec8-d33a-4e98-90a2-fe07abf01b5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 23 13:47:16.760451 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:16.760409 2531 patch_prober.go:28] interesting pod/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 23 13:47:16.760845 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:16.760472 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" podUID="27051ec8-d33a-4e98-90a2-fe07abf01b5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 23 13:47:17.764456 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:17.764426 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:47:17.764865 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:17.764708 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:47:17.765431 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:17.765411 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:47:32.696845 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:32.696815 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:47:37.622128 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:37.622096 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr"] Jul 23 13:47:37.622531 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:37.622294 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" podUID="89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" containerName="istio-proxy" containerID="cri-o://8ac390d5d0501c1db79b4875c8e5bcbe838b23d223f15fdfb80afbb42836e662" gracePeriod=30 Jul 23 13:47:42.451336 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:42.451293 2531 patch_prober.go:28] interesting pod/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 13:47:42.451751 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:42.451387 2531 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" podUID="89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 13:47:42.876585 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:42.876562 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:47:42.932931 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:42.932900 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istiod-ca-cert\") pod \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " Jul 23 13:47:42.933124 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:42.932949 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istio-podinfo\") pod \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " Jul 23 13:47:42.933124 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:42.932970 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istio-data\") pod \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " Jul 23 13:47:42.933124 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:42.932995 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-credential-socket\") pod \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " Jul 23 13:47:42.933124 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:42.933047 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-workload-socket\") pod \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " Jul 23 13:47:42.933124 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:42.933110 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwlpl\" (UniqueName: \"kubernetes.io/projected/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-kube-api-access-wwlpl\") pod \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " Jul 23 13:47:42.933421 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:42.933168 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istio-token\") pod \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " Jul 23 13:47:42.933421 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:42.933391 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" (UID: "89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:47:42.933421 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:42.933412 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" (UID: "89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:47:42.933585 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:42.933393 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" (UID: "89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:47:42.933585 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:42.933532 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-workload-certs\") pod \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " Jul 23 13:47:42.933698 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:42.933586 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istio-envoy\") pod \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\" (UID: \"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6\") " Jul 23 13:47:42.933999 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:42.933978 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" (UID: "89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:47:42.934161 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:42.934088 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istio-data" (OuterVolumeSpecName: "istio-data") pod "89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" (UID: "89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:47:42.934161 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:42.934149 2531 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istiod-ca-cert\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:47:42.934348 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:42.934168 2531 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-credential-socket\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:47:42.934348 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:42.934181 2531 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-workload-socket\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:47:42.935633 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:42.935607 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-kube-api-access-wwlpl" (OuterVolumeSpecName: "kube-api-access-wwlpl") pod "89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" (UID: "89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6"). InnerVolumeSpecName "kube-api-access-wwlpl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:47:42.935912 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:42.935887 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" (UID: "89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:47:42.936024 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:42.936006 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" (UID: "89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 13:47:42.936135 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:42.936122 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istio-token" (OuterVolumeSpecName: "istio-token") pod "89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" (UID: "89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:47:43.035527 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:43.035451 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwlpl\" (UniqueName: \"kubernetes.io/projected/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-kube-api-access-wwlpl\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:47:43.035527 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:43.035477 2531 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istio-token\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:47:43.035527 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:43.035489 2531 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-workload-certs\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:47:43.035527 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:43.035496 2531 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istio-envoy\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:47:43.035527 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:43.035503 2531 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istio-podinfo\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:47:43.035527 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:43.035510 2531 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6-istio-data\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:47:43.446832 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:43.446802 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6m8j_a0a5f6bc-bdb7-46ae-8cb9-79edd5179817/ovn-acl-logging/0.log" Jul 23 13:47:43.454876 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:43.454850 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6m8j_a0a5f6bc-bdb7-46ae-8cb9-79edd5179817/ovn-acl-logging/0.log" Jul 23 13:47:43.499502 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:43.499462 2531 generic.go:356] "Generic (PLEG): container finished" podID="89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" containerID="8ac390d5d0501c1db79b4875c8e5bcbe838b23d223f15fdfb80afbb42836e662" exitCode=0 Jul 23 13:47:43.499674 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:43.499529 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" Jul 23 13:47:43.499674 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:43.499550 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" event={"ID":"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6","Type":"ContainerDied","Data":"8ac390d5d0501c1db79b4875c8e5bcbe838b23d223f15fdfb80afbb42836e662"} Jul 23 13:47:43.499674 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:43.499630 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr" event={"ID":"89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6","Type":"ContainerDied","Data":"050741c355a0b33bb21a6abdf7b6b67e7b18114117dc5ba7a39325a75814534c"} Jul 23 13:47:43.499674 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:43.499652 2531 scope.go:117] "RemoveContainer" containerID="8ac390d5d0501c1db79b4875c8e5bcbe838b23d223f15fdfb80afbb42836e662" Jul 23 13:47:43.509118 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:43.509101 2531 scope.go:117] "RemoveContainer" containerID="338379ec5f2476d75a5df2549e4901f1f7ed1094801f4590fd3d5b39ff0a3815" Jul 23 13:47:43.518840 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:43.518817 2531 scope.go:117] "RemoveContainer" containerID="8ac390d5d0501c1db79b4875c8e5bcbe838b23d223f15fdfb80afbb42836e662" Jul 23 13:47:43.518958 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:43.518921 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr"] Jul 23 13:47:43.519179 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:47:43.519145 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ac390d5d0501c1db79b4875c8e5bcbe838b23d223f15fdfb80afbb42836e662\": container with ID starting with 8ac390d5d0501c1db79b4875c8e5bcbe838b23d223f15fdfb80afbb42836e662 not found: ID does not exist" containerID="8ac390d5d0501c1db79b4875c8e5bcbe838b23d223f15fdfb80afbb42836e662" Jul 23 13:47:43.519263 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:43.519178 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ac390d5d0501c1db79b4875c8e5bcbe838b23d223f15fdfb80afbb42836e662"} err="failed to get container status \"8ac390d5d0501c1db79b4875c8e5bcbe838b23d223f15fdfb80afbb42836e662\": rpc error: code = NotFound desc = could not find container \"8ac390d5d0501c1db79b4875c8e5bcbe838b23d223f15fdfb80afbb42836e662\": container with ID starting with 8ac390d5d0501c1db79b4875c8e5bcbe838b23d223f15fdfb80afbb42836e662 not found: ID does not exist" Jul 23 13:47:43.519263 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:43.519197 2531 scope.go:117] "RemoveContainer" containerID="338379ec5f2476d75a5df2549e4901f1f7ed1094801f4590fd3d5b39ff0a3815" Jul 23 13:47:43.519508 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:47:43.519488 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"338379ec5f2476d75a5df2549e4901f1f7ed1094801f4590fd3d5b39ff0a3815\": container with ID starting with 338379ec5f2476d75a5df2549e4901f1f7ed1094801f4590fd3d5b39ff0a3815 not found: ID does not exist" containerID="338379ec5f2476d75a5df2549e4901f1f7ed1094801f4590fd3d5b39ff0a3815" Jul 23 13:47:43.519577 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:43.519518 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"338379ec5f2476d75a5df2549e4901f1f7ed1094801f4590fd3d5b39ff0a3815"} err="failed to get container status \"338379ec5f2476d75a5df2549e4901f1f7ed1094801f4590fd3d5b39ff0a3815\": rpc error: code = NotFound desc = could not find container \"338379ec5f2476d75a5df2549e4901f1f7ed1094801f4590fd3d5b39ff0a3815\": container with ID starting with 338379ec5f2476d75a5df2549e4901f1f7ed1094801f4590fd3d5b39ff0a3815 not found: ID does not exist" Jul 23 13:47:43.521435 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:43.521410 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-636f5e-openshift-default-55fbb6f59c-kd4dr"] Jul 23 13:47:45.333276 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:45.333243 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" path="/var/lib/kubelet/pods/89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6/volumes" Jul 23 13:47:57.686770 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:47:57.686732 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:48:04.840085 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:04.840000 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh"] Jul 23 13:48:04.840523 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:04.840213 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" podUID="27051ec8-d33a-4e98-90a2-fe07abf01b5f" containerName="istio-proxy" containerID="cri-o://4e0827be1046d700eb1b9a23aebed83d9b410bf0706640c8f5fbc3d8474449eb" gracePeriod=30 Jul 23 13:48:05.844383 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:05.844338 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz"] Jul 23 13:48:05.844759 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:05.844740 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" containerName="istio-proxy" Jul 23 13:48:05.844759 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:05.844751 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" containerName="istio-proxy" Jul 23 13:48:05.844837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:05.844761 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" containerName="istio-proxy" Jul 23 13:48:05.844837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:05.844768 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" containerName="istio-proxy" Jul 23 13:48:05.844905 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:05.844844 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" containerName="istio-proxy" Jul 23 13:48:05.844905 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:05.844855 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="89ffc6c4-0f45-4ac4-aa19-8442c77d0eb6" containerName="istio-proxy" Jul 23 13:48:05.847900 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:05.847885 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:05.852190 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:05.852172 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-5ddc60-openshift-default-dockercfg-qpflr\"" Jul 23 13:48:05.881202 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:05.881176 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz"] Jul 23 13:48:05.943797 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:05.943755 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istio-data\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:05.943797 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:05.943801 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istiod-ca-cert\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:05.944026 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:05.943862 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-workload-socket\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:05.944026 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:05.943900 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-credential-socket\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:05.944026 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:05.943929 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-225w5\" (UniqueName: \"kubernetes.io/projected/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-kube-api-access-225w5\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:05.944026 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:05.943963 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istio-podinfo\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:05.944026 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:05.944007 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istio-envoy\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:05.944270 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:05.944042 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-workload-certs\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:05.944270 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:05.944107 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istio-token\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:06.045119 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.045086 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istio-data\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:06.045302 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.045128 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istiod-ca-cert\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:06.045302 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.045154 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-workload-socket\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:06.045302 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.045179 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-credential-socket\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:06.045302 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.045214 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-225w5\" (UniqueName: \"kubernetes.io/projected/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-kube-api-access-225w5\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:06.045302 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.045259 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istio-podinfo\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:06.045586 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.045341 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istio-envoy\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:06.045586 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.045420 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-workload-certs\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:06.045586 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.045482 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istio-token\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:06.045763 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.045674 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istio-data\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:06.045817 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.045760 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-workload-socket\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:06.045817 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.045780 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-credential-socket\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:06.045952 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.045932 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-workload-certs\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:06.046178 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.045987 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istiod-ca-cert\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:06.047674 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.047653 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istio-envoy\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:06.047933 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.047917 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istio-podinfo\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:06.059343 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.059317 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-225w5\" (UniqueName: \"kubernetes.io/projected/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-kube-api-access-225w5\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:06.059471 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.059342 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istio-token\") pod \"e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:06.157966 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.157884 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:06.311945 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.311919 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz"] Jul 23 13:48:06.313655 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:48:06.313623 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0a96c7c_b710_4efb_97c7_6f8e3b96811d.slice/crio-9fdfb19f532b20abc081e1b52c28386b121562b9d739edfabd52abae4c65db22 WatchSource:0}: Error finding container 9fdfb19f532b20abc081e1b52c28386b121562b9d739edfabd52abae4c65db22: Status 404 returned error can't find the container with id 9fdfb19f532b20abc081e1b52c28386b121562b9d739edfabd52abae4c65db22 Jul 23 13:48:06.315783 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.315744 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:48:06.315881 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.315823 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:48:06.315881 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.315858 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:48:06.630630 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.630589 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" event={"ID":"c0a96c7c-b710-4efb-97c7-6f8e3b96811d","Type":"ContainerStarted","Data":"5eb15b0c85768a6c2280aadfd9c246f208b9358ebff437e524278796ba631cbb"} Jul 23 13:48:06.630829 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.630647 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" event={"ID":"c0a96c7c-b710-4efb-97c7-6f8e3b96811d","Type":"ContainerStarted","Data":"9fdfb19f532b20abc081e1b52c28386b121562b9d739edfabd52abae4c65db22"} Jul 23 13:48:06.653648 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:06.653589 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" podStartSLOduration=1.6535698380000001 podStartE2EDuration="1.653569838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:48:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:48:06.651142555 +0000 UTC m=+2423.890418531" watchObservedRunningTime="2026-07-23 13:48:06.653569838 +0000 UTC m=+2423.892845819" Jul 23 13:48:07.158426 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:07.158389 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:07.163025 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:07.163001 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:07.636282 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:07.636245 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:07.637209 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:07.637187 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:10.090683 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.090659 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:48:10.185497 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.185408 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-workload-socket\") pod \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " Jul 23 13:48:10.185497 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.185454 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-credential-socket\") pod \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " Jul 23 13:48:10.185725 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.185513 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjlc9\" (UniqueName: \"kubernetes.io/projected/27051ec8-d33a-4e98-90a2-fe07abf01b5f-kube-api-access-vjlc9\") pod \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " Jul 23 13:48:10.185725 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.185551 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istio-data\") pod \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " Jul 23 13:48:10.185725 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.185585 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istio-envoy\") pod \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " Jul 23 13:48:10.185725 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.185609 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istiod-ca-cert\") pod \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " Jul 23 13:48:10.185725 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.185641 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istio-podinfo\") pod \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " Jul 23 13:48:10.185725 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.185690 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istio-token\") pod \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " Jul 23 13:48:10.186009 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.185734 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-workload-certs\") pod \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\" (UID: \"27051ec8-d33a-4e98-90a2-fe07abf01b5f\") " Jul 23 13:48:10.186009 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.185712 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "27051ec8-d33a-4e98-90a2-fe07abf01b5f" (UID: "27051ec8-d33a-4e98-90a2-fe07abf01b5f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:48:10.186009 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.185763 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "27051ec8-d33a-4e98-90a2-fe07abf01b5f" (UID: "27051ec8-d33a-4e98-90a2-fe07abf01b5f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:48:10.186180 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.186027 2531 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-workload-socket\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:48:10.186180 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.186045 2531 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-credential-socket\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:48:10.186180 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.186043 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "27051ec8-d33a-4e98-90a2-fe07abf01b5f" (UID: "27051ec8-d33a-4e98-90a2-fe07abf01b5f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:48:10.186331 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.186180 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "27051ec8-d33a-4e98-90a2-fe07abf01b5f" (UID: "27051ec8-d33a-4e98-90a2-fe07abf01b5f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:48:10.186826 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.186797 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istio-data" (OuterVolumeSpecName: "istio-data") pod "27051ec8-d33a-4e98-90a2-fe07abf01b5f" (UID: "27051ec8-d33a-4e98-90a2-fe07abf01b5f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:48:10.187919 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.187896 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "27051ec8-d33a-4e98-90a2-fe07abf01b5f" (UID: "27051ec8-d33a-4e98-90a2-fe07abf01b5f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:48:10.188014 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.187925 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "27051ec8-d33a-4e98-90a2-fe07abf01b5f" (UID: "27051ec8-d33a-4e98-90a2-fe07abf01b5f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 13:48:10.188014 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.187949 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istio-token" (OuterVolumeSpecName: "istio-token") pod "27051ec8-d33a-4e98-90a2-fe07abf01b5f" (UID: "27051ec8-d33a-4e98-90a2-fe07abf01b5f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:48:10.188014 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.187995 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27051ec8-d33a-4e98-90a2-fe07abf01b5f-kube-api-access-vjlc9" (OuterVolumeSpecName: "kube-api-access-vjlc9") pod "27051ec8-d33a-4e98-90a2-fe07abf01b5f" (UID: "27051ec8-d33a-4e98-90a2-fe07abf01b5f"). InnerVolumeSpecName "kube-api-access-vjlc9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:48:10.287562 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.287527 2531 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istio-data\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:48:10.287562 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.287555 2531 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istio-envoy\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:48:10.287562 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.287565 2531 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istiod-ca-cert\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:48:10.287562 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.287573 2531 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istio-podinfo\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:48:10.287821 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.287581 2531 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27051ec8-d33a-4e98-90a2-fe07abf01b5f-istio-token\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:48:10.287821 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.287590 2531 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27051ec8-d33a-4e98-90a2-fe07abf01b5f-workload-certs\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:48:10.287821 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.287597 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjlc9\" (UniqueName: \"kubernetes.io/projected/27051ec8-d33a-4e98-90a2-fe07abf01b5f-kube-api-access-vjlc9\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:48:10.653860 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.653826 2531 generic.go:356] "Generic (PLEG): container finished" podID="27051ec8-d33a-4e98-90a2-fe07abf01b5f" containerID="4e0827be1046d700eb1b9a23aebed83d9b410bf0706640c8f5fbc3d8474449eb" exitCode=0 Jul 23 13:48:10.654039 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.653873 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" Jul 23 13:48:10.654039 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.653891 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" event={"ID":"27051ec8-d33a-4e98-90a2-fe07abf01b5f","Type":"ContainerDied","Data":"4e0827be1046d700eb1b9a23aebed83d9b410bf0706640c8f5fbc3d8474449eb"} Jul 23 13:48:10.654039 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.653933 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh" event={"ID":"27051ec8-d33a-4e98-90a2-fe07abf01b5f","Type":"ContainerDied","Data":"cc40ba5014f26bf787ad7b2818fc335caa58f68bdb7458aa7ed7bb687ac54f73"} Jul 23 13:48:10.654039 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.653953 2531 scope.go:117] "RemoveContainer" containerID="4e0827be1046d700eb1b9a23aebed83d9b410bf0706640c8f5fbc3d8474449eb" Jul 23 13:48:10.663780 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.663762 2531 scope.go:117] "RemoveContainer" containerID="778f8fc3a2b89a0a3a9e4b8a5ef25382da871f1ce64daee726827e2733bfc724" Jul 23 13:48:10.672161 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.672142 2531 scope.go:117] "RemoveContainer" containerID="4e0827be1046d700eb1b9a23aebed83d9b410bf0706640c8f5fbc3d8474449eb" Jul 23 13:48:10.672426 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:48:10.672407 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e0827be1046d700eb1b9a23aebed83d9b410bf0706640c8f5fbc3d8474449eb\": container with ID starting with 4e0827be1046d700eb1b9a23aebed83d9b410bf0706640c8f5fbc3d8474449eb not found: ID does not exist" containerID="4e0827be1046d700eb1b9a23aebed83d9b410bf0706640c8f5fbc3d8474449eb" Jul 23 13:48:10.672478 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.672435 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e0827be1046d700eb1b9a23aebed83d9b410bf0706640c8f5fbc3d8474449eb"} err="failed to get container status \"4e0827be1046d700eb1b9a23aebed83d9b410bf0706640c8f5fbc3d8474449eb\": rpc error: code = NotFound desc = could not find container \"4e0827be1046d700eb1b9a23aebed83d9b410bf0706640c8f5fbc3d8474449eb\": container with ID starting with 4e0827be1046d700eb1b9a23aebed83d9b410bf0706640c8f5fbc3d8474449eb not found: ID does not exist" Jul 23 13:48:10.672478 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.672452 2531 scope.go:117] "RemoveContainer" containerID="778f8fc3a2b89a0a3a9e4b8a5ef25382da871f1ce64daee726827e2733bfc724" Jul 23 13:48:10.672707 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:48:10.672689 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"778f8fc3a2b89a0a3a9e4b8a5ef25382da871f1ce64daee726827e2733bfc724\": container with ID starting with 778f8fc3a2b89a0a3a9e4b8a5ef25382da871f1ce64daee726827e2733bfc724 not found: ID does not exist" containerID="778f8fc3a2b89a0a3a9e4b8a5ef25382da871f1ce64daee726827e2733bfc724" Jul 23 13:48:10.672753 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.672724 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"778f8fc3a2b89a0a3a9e4b8a5ef25382da871f1ce64daee726827e2733bfc724"} err="failed to get container status \"778f8fc3a2b89a0a3a9e4b8a5ef25382da871f1ce64daee726827e2733bfc724\": rpc error: code = NotFound desc = could not find container \"778f8fc3a2b89a0a3a9e4b8a5ef25382da871f1ce64daee726827e2733bfc724\": container with ID starting with 778f8fc3a2b89a0a3a9e4b8a5ef25382da871f1ce64daee726827e2733bfc724 not found: ID does not exist" Jul 23 13:48:10.676505 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.676476 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh"] Jul 23 13:48:10.680732 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:10.680712 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-a6a223-openshift-default-7ff6db66c-9xpdh"] Jul 23 13:48:11.332185 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:11.332150 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27051ec8-d33a-4e98-90a2-fe07abf01b5f" path="/var/lib/kubelet/pods/27051ec8-d33a-4e98-90a2-fe07abf01b5f/volumes" Jul 23 13:48:13.788511 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.788467 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p"] Jul 23 13:48:13.788921 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.788888 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27051ec8-d33a-4e98-90a2-fe07abf01b5f" containerName="istio-proxy" Jul 23 13:48:13.788921 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.788901 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="27051ec8-d33a-4e98-90a2-fe07abf01b5f" containerName="istio-proxy" Jul 23 13:48:13.789000 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.788966 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="27051ec8-d33a-4e98-90a2-fe07abf01b5f" containerName="istio-proxy" Jul 23 13:48:13.789000 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.788974 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="27051ec8-d33a-4e98-90a2-fe07abf01b5f" containerName="istio-proxy" Jul 23 13:48:13.793290 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.793273 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.796096 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.796079 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-690897-openshift-default-dockercfg-2rxbp\"" Jul 23 13:48:13.823370 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.823317 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istio-data\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.823614 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.823382 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istio-podinfo\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.823614 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.823441 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-workload-certs\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.823614 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.823480 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-credential-socket\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.823614 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.823538 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istio-token\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.823614 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.823561 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-workload-socket\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.823614 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.823600 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istiod-ca-cert\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.823814 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.823651 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istio-envoy\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.823814 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.823681 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcjgb\" (UniqueName: \"kubernetes.io/projected/81b9e02f-ad6a-4264-b7f9-06558ab5db66-kube-api-access-fcjgb\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.880193 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.880137 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p"] Jul 23 13:48:13.924837 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.924805 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istio-token\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.925034 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.924844 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-workload-socket\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.925034 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.924864 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istiod-ca-cert\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.925034 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.924892 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istio-envoy\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.925034 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.924925 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcjgb\" (UniqueName: \"kubernetes.io/projected/81b9e02f-ad6a-4264-b7f9-06558ab5db66-kube-api-access-fcjgb\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.925034 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.925002 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istio-data\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.925034 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.925022 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istio-podinfo\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.925348 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.925045 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-workload-certs\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.925348 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.925070 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-credential-socket\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.925348 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.925254 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-workload-socket\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.925575 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.925471 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-workload-certs\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.925575 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.925508 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istio-data\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.925575 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.925549 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-credential-socket\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.925687 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.925656 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istiod-ca-cert\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.927435 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.927411 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istio-envoy\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.927664 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.927642 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istio-podinfo\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.937114 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.937092 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcjgb\" (UniqueName: \"kubernetes.io/projected/81b9e02f-ad6a-4264-b7f9-06558ab5db66-kube-api-access-fcjgb\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:13.937329 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:13.937308 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istio-token\") pod \"e2e-shared-b-690897-openshift-default-57746579f5-kbr5p\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:14.103584 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:14.103554 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:14.258550 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:14.258524 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p"] Jul 23 13:48:14.260004 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:48:14.259956 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81b9e02f_ad6a_4264_b7f9_06558ab5db66.slice/crio-701d563f7a8bf35c7ab8b754ff82274163b39014e99133f3473d2311d6d2f6c6 WatchSource:0}: Error finding container 701d563f7a8bf35c7ab8b754ff82274163b39014e99133f3473d2311d6d2f6c6: Status 404 returned error can't find the container with id 701d563f7a8bf35c7ab8b754ff82274163b39014e99133f3473d2311d6d2f6c6 Jul 23 13:48:14.262298 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:14.262265 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:48:14.262423 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:14.262345 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:48:14.262423 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:14.262396 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:48:14.493313 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:14.493219 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:48:14.688700 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:14.688663 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" event={"ID":"81b9e02f-ad6a-4264-b7f9-06558ab5db66","Type":"ContainerStarted","Data":"2c2ea9d1f4f6f628632f4ce079b870c87282805bfa97e6759563b0aac1ca9804"} Jul 23 13:48:14.688876 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:14.688718 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" event={"ID":"81b9e02f-ad6a-4264-b7f9-06558ab5db66","Type":"ContainerStarted","Data":"701d563f7a8bf35c7ab8b754ff82274163b39014e99133f3473d2311d6d2f6c6"} Jul 23 13:48:14.715146 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:14.715083 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" podStartSLOduration=1.7150672500000002 podStartE2EDuration="1.71506725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:48:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:48:14.711773302 +0000 UTC m=+2431.951049278" watchObservedRunningTime="2026-07-23 13:48:14.71506725 +0000 UTC m=+2431.954343225" Jul 23 13:48:15.104042 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:15.104000 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:15.109481 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:15.109451 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:15.693849 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:15.693813 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:15.694794 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:15.694775 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:22.637648 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:22.637609 2531 patch_prober.go:28] interesting pod/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 23 13:48:22.638051 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:22.637673 2531 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" podUID="c0a96c7c-b710-4efb-97c7-6f8e3b96811d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 23 13:48:22.736221 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:22.736190 2531 generic.go:356] "Generic (PLEG): container finished" podID="c0a96c7c-b710-4efb-97c7-6f8e3b96811d" containerID="5eb15b0c85768a6c2280aadfd9c246f208b9358ebff437e524278796ba631cbb" exitCode=0 Jul 23 13:48:22.736418 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:22.736268 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" event={"ID":"c0a96c7c-b710-4efb-97c7-6f8e3b96811d","Type":"ContainerDied","Data":"5eb15b0c85768a6c2280aadfd9c246f208b9358ebff437e524278796ba631cbb"} Jul 23 13:48:22.736750 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:22.736735 2531 scope.go:117] "RemoveContainer" containerID="5eb15b0c85768a6c2280aadfd9c246f208b9358ebff437e524278796ba631cbb" Jul 23 13:48:22.737755 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:22.737726 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:48:22.737863 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:22.737831 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:48:22.737993 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:22.737880 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:48:22.897460 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:22.897374 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:48:23.748595 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:23.748544 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" event={"ID":"c0a96c7c-b710-4efb-97c7-6f8e3b96811d","Type":"ContainerStarted","Data":"27c729129f97e2b599f430917283e1da244af284ccce0864445f7d47f895143c"} Jul 23 13:48:24.158964 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:24.158914 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:24.159188 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:24.159056 2531 patch_prober.go:28] interesting pod/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 23 13:48:24.159188 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:24.159113 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" podUID="c0a96c7c-b710-4efb-97c7-6f8e3b96811d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 23 13:48:25.158722 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:25.158678 2531 patch_prober.go:28] interesting pod/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 23 13:48:25.159216 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:25.158747 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" podUID="c0a96c7c-b710-4efb-97c7-6f8e3b96811d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 23 13:48:26.159096 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:26.159046 2531 patch_prober.go:28] interesting pod/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 23 13:48:26.159660 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:26.159122 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" podUID="c0a96c7c-b710-4efb-97c7-6f8e3b96811d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 23 13:48:27.158721 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:27.158678 2531 patch_prober.go:28] interesting pod/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 23 13:48:27.158957 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:27.158752 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" podUID="c0a96c7c-b710-4efb-97c7-6f8e3b96811d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 23 13:48:28.163348 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:28.163311 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:28.163793 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:28.163738 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:28.164593 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:28.164575 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:48:30.770326 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:30.769266 2531 patch_prober.go:28] interesting pod/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": read tcp 10.134.0.2:55950->10.134.0.64:15021: read: connection reset by peer" start-of-body= Jul 23 13:48:30.770326 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:30.769339 2531 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" podUID="81b9e02f-ad6a-4264-b7f9-06558ab5db66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": read tcp 10.134.0.2:55950->10.134.0.64:15021: read: connection reset by peer" Jul 23 13:48:31.807110 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:31.807075 2531 generic.go:356] "Generic (PLEG): container finished" podID="81b9e02f-ad6a-4264-b7f9-06558ab5db66" containerID="2c2ea9d1f4f6f628632f4ce079b870c87282805bfa97e6759563b0aac1ca9804" exitCode=0 Jul 23 13:48:31.807507 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:31.807151 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" event={"ID":"81b9e02f-ad6a-4264-b7f9-06558ab5db66","Type":"ContainerDied","Data":"2c2ea9d1f4f6f628632f4ce079b870c87282805bfa97e6759563b0aac1ca9804"} Jul 23 13:48:31.807652 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:31.807634 2531 scope.go:117] "RemoveContainer" containerID="2c2ea9d1f4f6f628632f4ce079b870c87282805bfa97e6759563b0aac1ca9804" Jul 23 13:48:31.808668 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:31.808635 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:48:31.808806 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:31.808724 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:48:31.808806 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:31.808774 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 23 13:48:32.814562 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:32.814524 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" event={"ID":"81b9e02f-ad6a-4264-b7f9-06558ab5db66","Type":"ContainerStarted","Data":"9a0c571154fa3bd595fc174da831de73d1cf25e9a4092bb6e635de3f9a4b7e2a"} Jul 23 13:48:33.104772 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:33.104663 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:33.104955 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:33.104839 2531 patch_prober.go:28] interesting pod/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Jul 23 13:48:33.104955 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:33.104894 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" podUID="81b9e02f-ad6a-4264-b7f9-06558ab5db66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Jul 23 13:48:34.103965 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:34.103928 2531 patch_prober.go:28] interesting pod/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Jul 23 13:48:34.104438 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:34.103983 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" podUID="81b9e02f-ad6a-4264-b7f9-06558ab5db66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Jul 23 13:48:35.104427 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:35.104386 2531 patch_prober.go:28] interesting pod/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Jul 23 13:48:35.104840 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:35.104446 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" podUID="81b9e02f-ad6a-4264-b7f9-06558ab5db66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Jul 23 13:48:36.104099 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:36.104056 2531 patch_prober.go:28] interesting pod/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Jul 23 13:48:36.104284 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:36.104120 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" podUID="81b9e02f-ad6a-4264-b7f9-06558ab5db66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Jul 23 13:48:37.108155 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:37.108122 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:37.108563 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:37.108466 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:37.109041 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:37.109022 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:48:40.494216 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:40.494175 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:48:43.622141 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:48:43.622108 2531 scope.go:117] "RemoveContainer" containerID="ea5e7a780e50754509926f18843a1f43be021298dfa627f03bfab6040a2530a9" Jul 23 13:49:17.594162 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:49:17.594128 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:49:45.691646 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:49:45.691562 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:50:01.182439 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:50:01.182402 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:50:19.185084 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:50:19.185049 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:50:36.305515 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:50:36.305478 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:50:52.888497 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:50:52.888453 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:51:11.400740 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:51:11.400705 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:51:29.294306 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:51:29.294276 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:51:39.100117 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:51:39.100079 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:51:43.668983 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:51:43.668942 2531 scope.go:117] "RemoveContainer" containerID="24854a0c083f803f72a3da321441cfcbec27b76fecca5a25d05cb5bd40de488b" Jul 23 13:51:59.489973 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:51:59.489933 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:52:18.489608 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:52:18.489574 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:52:32.098589 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:52:32.098550 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:52:43.486963 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:52:43.486883 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6m8j_a0a5f6bc-bdb7-46ae-8cb9-79edd5179817/ovn-acl-logging/0.log" Jul 23 13:52:43.493244 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:52:43.493219 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6m8j_a0a5f6bc-bdb7-46ae-8cb9-79edd5179817/ovn-acl-logging/0.log" Jul 23 13:52:48.609119 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:52:48.609081 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:53:18.286716 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:53:18.286677 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:53:37.481686 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:53:37.481647 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:53:52.398042 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:53:52.398004 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:54:01.414841 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:01.414802 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7241-139755630892336"] Jul 23 13:54:01.415344 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:01.415195 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27051ec8-d33a-4e98-90a2-fe07abf01b5f" containerName="istio-proxy" Jul 23 13:54:01.415344 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:01.415206 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="27051ec8-d33a-4e98-90a2-fe07abf01b5f" containerName="istio-proxy" Jul 23 13:54:01.418258 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:01.418237 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7241-139755630892336" Jul 23 13:54:01.420813 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:01.420789 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-9rv6n\"" Jul 23 13:54:01.424479 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:01.424444 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7241-139755630892336"] Jul 23 13:54:01.529107 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:01.529071 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6n72\" (UniqueName: \"kubernetes.io/projected/a1397487-469a-489e-bc4b-3859512cb398-kube-api-access-w6n72\") pod \"test-curl-7241-139755630892336\" (UID: \"a1397487-469a-489e-bc4b-3859512cb398\") " pod="opendatahub/test-curl-7241-139755630892336" Jul 23 13:54:01.630057 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:01.630018 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6n72\" (UniqueName: \"kubernetes.io/projected/a1397487-469a-489e-bc4b-3859512cb398-kube-api-access-w6n72\") pod \"test-curl-7241-139755630892336\" (UID: \"a1397487-469a-489e-bc4b-3859512cb398\") " pod="opendatahub/test-curl-7241-139755630892336" Jul 23 13:54:01.639651 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:01.639625 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6n72\" (UniqueName: \"kubernetes.io/projected/a1397487-469a-489e-bc4b-3859512cb398-kube-api-access-w6n72\") pod \"test-curl-7241-139755630892336\" (UID: \"a1397487-469a-489e-bc4b-3859512cb398\") " pod="opendatahub/test-curl-7241-139755630892336" Jul 23 13:54:01.730954 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:01.730857 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7241-139755630892336" Jul 23 13:54:01.873668 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:01.873643 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7241-139755630892336"] Jul 23 13:54:01.874951 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:54:01.874925 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1397487_469a_489e_bc4b_3859512cb398.slice/crio-1be6a88733d3857dc636797f1e32dca366ef94f506945b163174ba0758a980e7 WatchSource:0}: Error finding container 1be6a88733d3857dc636797f1e32dca366ef94f506945b163174ba0758a980e7: Status 404 returned error can't find the container with id 1be6a88733d3857dc636797f1e32dca366ef94f506945b163174ba0758a980e7 Jul 23 13:54:01.876208 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:01.876189 2531 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 13:54:02.669559 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:02.669026 2531 generic.go:356] "Generic (PLEG): container finished" podID="a1397487-469a-489e-bc4b-3859512cb398" containerID="e212d56ac85680ff77f68a0bc8853493b110b2dbd76de596c1e63d358e04d92a" exitCode=0 Jul 23 13:54:02.669559 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:02.669207 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7241-139755630892336" event={"ID":"a1397487-469a-489e-bc4b-3859512cb398","Type":"ContainerDied","Data":"e212d56ac85680ff77f68a0bc8853493b110b2dbd76de596c1e63d358e04d92a"} Jul 23 13:54:02.669559 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:02.669244 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7241-139755630892336" event={"ID":"a1397487-469a-489e-bc4b-3859512cb398","Type":"ContainerStarted","Data":"1be6a88733d3857dc636797f1e32dca366ef94f506945b163174ba0758a980e7"} Jul 23 13:54:03.808048 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:03.808027 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7241-139755630892336" Jul 23 13:54:03.845948 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:03.845914 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7241-139755630892336_a1397487-469a-489e-bc4b-3859512cb398/test-curl-7241-139755630892336/0.log" Jul 23 13:54:03.852621 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:03.852584 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6n72\" (UniqueName: \"kubernetes.io/projected/a1397487-469a-489e-bc4b-3859512cb398-kube-api-access-w6n72\") pod \"a1397487-469a-489e-bc4b-3859512cb398\" (UID: \"a1397487-469a-489e-bc4b-3859512cb398\") " Jul 23 13:54:03.854857 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:03.854835 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1397487-469a-489e-bc4b-3859512cb398-kube-api-access-w6n72" (OuterVolumeSpecName: "kube-api-access-w6n72") pod "a1397487-469a-489e-bc4b-3859512cb398" (UID: "a1397487-469a-489e-bc4b-3859512cb398"). InnerVolumeSpecName "kube-api-access-w6n72". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:54:03.871319 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:03.871293 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7241-139755630892336"] Jul 23 13:54:03.875277 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:03.875259 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7241-139755630892336"] Jul 23 13:54:03.954450 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:03.954322 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6n72\" (UniqueName: \"kubernetes.io/projected/a1397487-469a-489e-bc4b-3859512cb398-kube-api-access-w6n72\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:54:04.680868 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:04.680827 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7241-139755630892336" Jul 23 13:54:04.680868 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:04.680863 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1be6a88733d3857dc636797f1e32dca366ef94f506945b163174ba0758a980e7" Jul 23 13:54:04.690699 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:54:04.690667 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7241-139755630892336\" is forbidden: User \"system:node:ip-10-0-130-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-1.ec2.internal' and this object" podUID="a1397487-469a-489e-bc4b-3859512cb398" pod="opendatahub/test-curl-7241-139755630892336" Jul 23 13:54:05.332875 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:05.332840 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1397487-469a-489e-bc4b-3859512cb398" path="/var/lib/kubelet/pods/a1397487-469a-489e-bc4b-3859512cb398/volumes" Jul 23 13:54:12.587871 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:12.587819 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:54:13.055897 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:13.055868 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7241-139755629743536"] Jul 23 13:54:13.056281 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:13.056268 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1397487-469a-489e-bc4b-3859512cb398" containerName="test-curl-7241-139755630892336" Jul 23 13:54:13.056321 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:13.056282 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1397487-469a-489e-bc4b-3859512cb398" containerName="test-curl-7241-139755630892336" Jul 23 13:54:13.056384 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:13.056374 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1397487-469a-489e-bc4b-3859512cb398" containerName="test-curl-7241-139755630892336" Jul 23 13:54:13.059565 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:13.059549 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7241-139755629743536" Jul 23 13:54:13.062171 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:13.062149 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-9rv6n\"" Jul 23 13:54:13.064556 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:13.064529 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7241-139755629743536"] Jul 23 13:54:13.147668 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:13.147633 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss57n\" (UniqueName: \"kubernetes.io/projected/f5f9167f-50fb-41c8-bf8d-e2781ee44633-kube-api-access-ss57n\") pod \"test-curl-7241-139755629743536\" (UID: \"f5f9167f-50fb-41c8-bf8d-e2781ee44633\") " pod="opendatahub/test-curl-7241-139755629743536" Jul 23 13:54:13.248227 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:13.248194 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss57n\" (UniqueName: \"kubernetes.io/projected/f5f9167f-50fb-41c8-bf8d-e2781ee44633-kube-api-access-ss57n\") pod \"test-curl-7241-139755629743536\" (UID: \"f5f9167f-50fb-41c8-bf8d-e2781ee44633\") " pod="opendatahub/test-curl-7241-139755629743536" Jul 23 13:54:13.257347 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:13.257326 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss57n\" (UniqueName: \"kubernetes.io/projected/f5f9167f-50fb-41c8-bf8d-e2781ee44633-kube-api-access-ss57n\") pod \"test-curl-7241-139755629743536\" (UID: \"f5f9167f-50fb-41c8-bf8d-e2781ee44633\") " pod="opendatahub/test-curl-7241-139755629743536" Jul 23 13:54:13.370027 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:13.369947 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7241-139755629743536" Jul 23 13:54:13.508860 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:13.508825 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7241-139755629743536"] Jul 23 13:54:13.510402 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:54:13.510376 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf5f9167f_50fb_41c8_bf8d_e2781ee44633.slice/crio-ea19991a8a640aaf1eb5eccd8d95de78b3004526c313f7cba19981e637ff2b65 WatchSource:0}: Error finding container ea19991a8a640aaf1eb5eccd8d95de78b3004526c313f7cba19981e637ff2b65: Status 404 returned error can't find the container with id ea19991a8a640aaf1eb5eccd8d95de78b3004526c313f7cba19981e637ff2b65 Jul 23 13:54:13.733842 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:13.733753 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7241-139755629743536" event={"ID":"f5f9167f-50fb-41c8-bf8d-e2781ee44633","Type":"ContainerStarted","Data":"ea19991a8a640aaf1eb5eccd8d95de78b3004526c313f7cba19981e637ff2b65"} Jul 23 13:54:14.740733 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:14.740695 2531 generic.go:356] "Generic (PLEG): container finished" podID="f5f9167f-50fb-41c8-bf8d-e2781ee44633" containerID="abf19c64e7f7a2086032895049beadd5d1580bc8accaa42b690e7638bdb17c67" exitCode=0 Jul 23 13:54:14.741110 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:14.740753 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7241-139755629743536" event={"ID":"f5f9167f-50fb-41c8-bf8d-e2781ee44633","Type":"ContainerDied","Data":"abf19c64e7f7a2086032895049beadd5d1580bc8accaa42b690e7638bdb17c67"} Jul 23 13:54:15.891012 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:15.890987 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7241-139755629743536" Jul 23 13:54:15.916736 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:15.916710 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7241-139755629743536_f5f9167f-50fb-41c8-bf8d-e2781ee44633/test-curl-7241-139755629743536/0.log" Jul 23 13:54:15.944314 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:15.944274 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7241-139755629743536"] Jul 23 13:54:15.949646 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:15.949621 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7241-139755629743536"] Jul 23 13:54:15.977436 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:15.977410 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ss57n\" (UniqueName: \"kubernetes.io/projected/f5f9167f-50fb-41c8-bf8d-e2781ee44633-kube-api-access-ss57n\") pod \"f5f9167f-50fb-41c8-bf8d-e2781ee44633\" (UID: \"f5f9167f-50fb-41c8-bf8d-e2781ee44633\") " Jul 23 13:54:15.979550 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:15.979527 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5f9167f-50fb-41c8-bf8d-e2781ee44633-kube-api-access-ss57n" (OuterVolumeSpecName: "kube-api-access-ss57n") pod "f5f9167f-50fb-41c8-bf8d-e2781ee44633" (UID: "f5f9167f-50fb-41c8-bf8d-e2781ee44633"). InnerVolumeSpecName "kube-api-access-ss57n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:54:16.078174 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:16.078139 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ss57n\" (UniqueName: \"kubernetes.io/projected/f5f9167f-50fb-41c8-bf8d-e2781ee44633-kube-api-access-ss57n\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:54:16.752619 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:16.752585 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7241-139755629743536" Jul 23 13:54:16.752798 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:16.752626 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea19991a8a640aaf1eb5eccd8d95de78b3004526c313f7cba19981e637ff2b65" Jul 23 13:54:16.762369 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:54:16.762329 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7241-139755629743536\" is forbidden: User \"system:node:ip-10-0-130-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-1.ec2.internal' and this object" podUID="f5f9167f-50fb-41c8-bf8d-e2781ee44633" pod="opendatahub/test-curl-7241-139755629743536" Jul 23 13:54:17.333153 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:17.333121 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5f9167f-50fb-41c8-bf8d-e2781ee44633" path="/var/lib/kubelet/pods/f5f9167f-50fb-41c8-bf8d-e2781ee44633/volumes" Jul 23 13:54:33.294484 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:33.294387 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:54:40.980303 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:40.980266 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz"] Jul 23 13:54:40.980792 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:40.980564 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" podUID="c0a96c7c-b710-4efb-97c7-6f8e3b96811d" containerName="istio-proxy" containerID="cri-o://27c729129f97e2b599f430917283e1da244af284ccce0864445f7d47f895143c" gracePeriod=30 Jul 23 13:54:43.166299 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:43.166265 2531 patch_prober.go:28] interesting pod/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 13:54:43.166692 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:43.166332 2531 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" podUID="c0a96c7c-b710-4efb-97c7-6f8e3b96811d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 13:54:46.236686 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.236663 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:54:46.357937 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.357908 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-workload-certs\") pod \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " Jul 23 13:54:46.358094 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.357958 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-credential-socket\") pod \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " Jul 23 13:54:46.358094 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.358018 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istio-envoy\") pod \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " Jul 23 13:54:46.358094 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.358043 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istio-podinfo\") pod \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " Jul 23 13:54:46.358247 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.358101 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istio-data\") pod \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " Jul 23 13:54:46.358247 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.358130 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istio-token\") pod \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " Jul 23 13:54:46.358247 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.358174 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-workload-socket\") pod \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " Jul 23 13:54:46.358247 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.358224 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-225w5\" (UniqueName: \"kubernetes.io/projected/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-kube-api-access-225w5\") pod \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " Jul 23 13:54:46.358484 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.358253 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istiod-ca-cert\") pod \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\" (UID: \"c0a96c7c-b710-4efb-97c7-6f8e3b96811d\") " Jul 23 13:54:46.358484 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.358257 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c0a96c7c-b710-4efb-97c7-6f8e3b96811d" (UID: "c0a96c7c-b710-4efb-97c7-6f8e3b96811d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:54:46.358484 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.358386 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c0a96c7c-b710-4efb-97c7-6f8e3b96811d" (UID: "c0a96c7c-b710-4efb-97c7-6f8e3b96811d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:54:46.358648 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.358495 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c0a96c7c-b710-4efb-97c7-6f8e3b96811d" (UID: "c0a96c7c-b710-4efb-97c7-6f8e3b96811d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:54:46.358703 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.358664 2531 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-workload-socket\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:54:46.358703 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.358682 2531 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-workload-certs\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:54:46.358703 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.358695 2531 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-credential-socket\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:54:46.358852 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.358699 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c0a96c7c-b710-4efb-97c7-6f8e3b96811d" (UID: "c0a96c7c-b710-4efb-97c7-6f8e3b96811d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:54:46.359076 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.359051 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istio-data" (OuterVolumeSpecName: "istio-data") pod "c0a96c7c-b710-4efb-97c7-6f8e3b96811d" (UID: "c0a96c7c-b710-4efb-97c7-6f8e3b96811d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:54:46.360561 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.360534 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c0a96c7c-b710-4efb-97c7-6f8e3b96811d" (UID: "c0a96c7c-b710-4efb-97c7-6f8e3b96811d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 13:54:46.360667 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.360595 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-kube-api-access-225w5" (OuterVolumeSpecName: "kube-api-access-225w5") pod "c0a96c7c-b710-4efb-97c7-6f8e3b96811d" (UID: "c0a96c7c-b710-4efb-97c7-6f8e3b96811d"). InnerVolumeSpecName "kube-api-access-225w5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:54:46.360993 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.360973 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c0a96c7c-b710-4efb-97c7-6f8e3b96811d" (UID: "c0a96c7c-b710-4efb-97c7-6f8e3b96811d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:54:46.361056 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.360994 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istio-token" (OuterVolumeSpecName: "istio-token") pod "c0a96c7c-b710-4efb-97c7-6f8e3b96811d" (UID: "c0a96c7c-b710-4efb-97c7-6f8e3b96811d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:54:46.460848 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.460799 2531 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istio-envoy\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:54:46.460848 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.460840 2531 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istio-podinfo\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:54:46.460848 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.460852 2531 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istio-data\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:54:46.461139 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.460870 2531 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istio-token\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:54:46.461139 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.460882 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-225w5\" (UniqueName: \"kubernetes.io/projected/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-kube-api-access-225w5\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:54:46.461139 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.460893 2531 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0a96c7c-b710-4efb-97c7-6f8e3b96811d-istiod-ca-cert\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:54:46.925437 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.925399 2531 generic.go:356] "Generic (PLEG): container finished" podID="c0a96c7c-b710-4efb-97c7-6f8e3b96811d" containerID="27c729129f97e2b599f430917283e1da244af284ccce0864445f7d47f895143c" exitCode=0 Jul 23 13:54:46.925609 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.925462 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" Jul 23 13:54:46.925609 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.925487 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" event={"ID":"c0a96c7c-b710-4efb-97c7-6f8e3b96811d","Type":"ContainerDied","Data":"27c729129f97e2b599f430917283e1da244af284ccce0864445f7d47f895143c"} Jul 23 13:54:46.925609 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.925567 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz" event={"ID":"c0a96c7c-b710-4efb-97c7-6f8e3b96811d","Type":"ContainerDied","Data":"9fdfb19f532b20abc081e1b52c28386b121562b9d739edfabd52abae4c65db22"} Jul 23 13:54:46.925609 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.925590 2531 scope.go:117] "RemoveContainer" containerID="27c729129f97e2b599f430917283e1da244af284ccce0864445f7d47f895143c" Jul 23 13:54:46.935504 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.935488 2531 scope.go:117] "RemoveContainer" containerID="5eb15b0c85768a6c2280aadfd9c246f208b9358ebff437e524278796ba631cbb" Jul 23 13:54:46.943704 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.943686 2531 scope.go:117] "RemoveContainer" containerID="27c729129f97e2b599f430917283e1da244af284ccce0864445f7d47f895143c" Jul 23 13:54:46.943962 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:54:46.943943 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27c729129f97e2b599f430917283e1da244af284ccce0864445f7d47f895143c\": container with ID starting with 27c729129f97e2b599f430917283e1da244af284ccce0864445f7d47f895143c not found: ID does not exist" containerID="27c729129f97e2b599f430917283e1da244af284ccce0864445f7d47f895143c" Jul 23 13:54:46.944014 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.943972 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27c729129f97e2b599f430917283e1da244af284ccce0864445f7d47f895143c"} err="failed to get container status \"27c729129f97e2b599f430917283e1da244af284ccce0864445f7d47f895143c\": rpc error: code = NotFound desc = could not find container \"27c729129f97e2b599f430917283e1da244af284ccce0864445f7d47f895143c\": container with ID starting with 27c729129f97e2b599f430917283e1da244af284ccce0864445f7d47f895143c not found: ID does not exist" Jul 23 13:54:46.944014 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.943989 2531 scope.go:117] "RemoveContainer" containerID="5eb15b0c85768a6c2280aadfd9c246f208b9358ebff437e524278796ba631cbb" Jul 23 13:54:46.944232 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:54:46.944216 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5eb15b0c85768a6c2280aadfd9c246f208b9358ebff437e524278796ba631cbb\": container with ID starting with 5eb15b0c85768a6c2280aadfd9c246f208b9358ebff437e524278796ba631cbb not found: ID does not exist" containerID="5eb15b0c85768a6c2280aadfd9c246f208b9358ebff437e524278796ba631cbb" Jul 23 13:54:46.944268 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.944239 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eb15b0c85768a6c2280aadfd9c246f208b9358ebff437e524278796ba631cbb"} err="failed to get container status \"5eb15b0c85768a6c2280aadfd9c246f208b9358ebff437e524278796ba631cbb\": rpc error: code = NotFound desc = could not find container \"5eb15b0c85768a6c2280aadfd9c246f208b9358ebff437e524278796ba631cbb\": container with ID starting with 5eb15b0c85768a6c2280aadfd9c246f208b9358ebff437e524278796ba631cbb not found: ID does not exist" Jul 23 13:54:46.950256 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.950230 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz"] Jul 23 13:54:46.956871 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:46.956851 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-5ddc60-openshift-default-5499d59cc5-plvmz"] Jul 23 13:54:47.332481 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:47.332438 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0a96c7c-b710-4efb-97c7-6f8e3b96811d" path="/var/lib/kubelet/pods/c0a96c7c-b710-4efb-97c7-6f8e3b96811d/volumes" Jul 23 13:54:54.586802 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:54:54.586769 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4xtb"] Jul 23 13:55:03.102412 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:03.102374 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p"] Jul 23 13:55:03.102852 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:03.102602 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" podUID="81b9e02f-ad6a-4264-b7f9-06558ab5db66" containerName="istio-proxy" containerID="cri-o://9a0c571154fa3bd595fc174da831de73d1cf25e9a4092bb6e635de3f9a4b7e2a" gracePeriod=30 Jul 23 13:55:07.110749 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:07.110714 2531 patch_prober.go:28] interesting pod/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 13:55:07.111124 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:07.110772 2531 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" podUID="81b9e02f-ad6a-4264-b7f9-06558ab5db66" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 13:55:08.360013 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.359986 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:55:08.464678 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.464594 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istio-envoy\") pod \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " Jul 23 13:55:08.464678 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.464664 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istiod-ca-cert\") pod \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " Jul 23 13:55:08.464921 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.464690 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-workload-certs\") pod \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " Jul 23 13:55:08.464921 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.464744 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istio-data\") pod \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " Jul 23 13:55:08.464921 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.464884 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcjgb\" (UniqueName: \"kubernetes.io/projected/81b9e02f-ad6a-4264-b7f9-06558ab5db66-kube-api-access-fcjgb\") pod \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " Jul 23 13:55:08.465079 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.464941 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istio-podinfo\") pod \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " Jul 23 13:55:08.465079 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.464977 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-workload-socket\") pod \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " Jul 23 13:55:08.465079 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.465011 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-credential-socket\") pod \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " Jul 23 13:55:08.465079 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.465009 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "81b9e02f-ad6a-4264-b7f9-06558ab5db66" (UID: "81b9e02f-ad6a-4264-b7f9-06558ab5db66"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:55:08.465079 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.465042 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istio-token\") pod \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\" (UID: \"81b9e02f-ad6a-4264-b7f9-06558ab5db66\") " Jul 23 13:55:08.465338 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.465082 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "81b9e02f-ad6a-4264-b7f9-06558ab5db66" (UID: "81b9e02f-ad6a-4264-b7f9-06558ab5db66"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:55:08.465338 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.465263 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "81b9e02f-ad6a-4264-b7f9-06558ab5db66" (UID: "81b9e02f-ad6a-4264-b7f9-06558ab5db66"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:55:08.465338 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.465277 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "81b9e02f-ad6a-4264-b7f9-06558ab5db66" (UID: "81b9e02f-ad6a-4264-b7f9-06558ab5db66"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:55:08.465510 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.465431 2531 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-workload-socket\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:55:08.465510 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.465452 2531 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-credential-socket\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:55:08.465510 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.465466 2531 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istiod-ca-cert\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:55:08.465510 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.465479 2531 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-workload-certs\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:55:08.466302 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.466279 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istio-data" (OuterVolumeSpecName: "istio-data") pod "81b9e02f-ad6a-4264-b7f9-06558ab5db66" (UID: "81b9e02f-ad6a-4264-b7f9-06558ab5db66"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:55:08.467296 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.467272 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81b9e02f-ad6a-4264-b7f9-06558ab5db66-kube-api-access-fcjgb" (OuterVolumeSpecName: "kube-api-access-fcjgb") pod "81b9e02f-ad6a-4264-b7f9-06558ab5db66" (UID: "81b9e02f-ad6a-4264-b7f9-06558ab5db66"). InnerVolumeSpecName "kube-api-access-fcjgb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:55:08.467374 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.467290 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istio-token" (OuterVolumeSpecName: "istio-token") pod "81b9e02f-ad6a-4264-b7f9-06558ab5db66" (UID: "81b9e02f-ad6a-4264-b7f9-06558ab5db66"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:55:08.467546 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.467525 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "81b9e02f-ad6a-4264-b7f9-06558ab5db66" (UID: "81b9e02f-ad6a-4264-b7f9-06558ab5db66"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 13:55:08.467613 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.467542 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "81b9e02f-ad6a-4264-b7f9-06558ab5db66" (UID: "81b9e02f-ad6a-4264-b7f9-06558ab5db66"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:55:08.566449 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.566417 2531 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istio-token\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:55:08.566449 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.566446 2531 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istio-envoy\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:55:08.566449 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.566454 2531 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istio-data\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:55:08.566687 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.566462 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcjgb\" (UniqueName: \"kubernetes.io/projected/81b9e02f-ad6a-4264-b7f9-06558ab5db66-kube-api-access-fcjgb\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:55:08.566687 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:08.566471 2531 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81b9e02f-ad6a-4264-b7f9-06558ab5db66-istio-podinfo\") on node \"ip-10-0-130-1.ec2.internal\" DevicePath \"\"" Jul 23 13:55:09.060120 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:09.060088 2531 generic.go:356] "Generic (PLEG): container finished" podID="81b9e02f-ad6a-4264-b7f9-06558ab5db66" containerID="9a0c571154fa3bd595fc174da831de73d1cf25e9a4092bb6e635de3f9a4b7e2a" exitCode=0 Jul 23 13:55:09.060303 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:09.060150 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" Jul 23 13:55:09.060303 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:09.060164 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" event={"ID":"81b9e02f-ad6a-4264-b7f9-06558ab5db66","Type":"ContainerDied","Data":"9a0c571154fa3bd595fc174da831de73d1cf25e9a4092bb6e635de3f9a4b7e2a"} Jul 23 13:55:09.060303 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:09.060208 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p" event={"ID":"81b9e02f-ad6a-4264-b7f9-06558ab5db66","Type":"ContainerDied","Data":"701d563f7a8bf35c7ab8b754ff82274163b39014e99133f3473d2311d6d2f6c6"} Jul 23 13:55:09.060303 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:09.060230 2531 scope.go:117] "RemoveContainer" containerID="9a0c571154fa3bd595fc174da831de73d1cf25e9a4092bb6e635de3f9a4b7e2a" Jul 23 13:55:09.070057 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:09.070037 2531 scope.go:117] "RemoveContainer" containerID="2c2ea9d1f4f6f628632f4ce079b870c87282805bfa97e6759563b0aac1ca9804" Jul 23 13:55:09.078848 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:09.078828 2531 scope.go:117] "RemoveContainer" containerID="9a0c571154fa3bd595fc174da831de73d1cf25e9a4092bb6e635de3f9a4b7e2a" Jul 23 13:55:09.079104 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:55:09.079084 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a0c571154fa3bd595fc174da831de73d1cf25e9a4092bb6e635de3f9a4b7e2a\": container with ID starting with 9a0c571154fa3bd595fc174da831de73d1cf25e9a4092bb6e635de3f9a4b7e2a not found: ID does not exist" containerID="9a0c571154fa3bd595fc174da831de73d1cf25e9a4092bb6e635de3f9a4b7e2a" Jul 23 13:55:09.079175 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:09.079116 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a0c571154fa3bd595fc174da831de73d1cf25e9a4092bb6e635de3f9a4b7e2a"} err="failed to get container status \"9a0c571154fa3bd595fc174da831de73d1cf25e9a4092bb6e635de3f9a4b7e2a\": rpc error: code = NotFound desc = could not find container \"9a0c571154fa3bd595fc174da831de73d1cf25e9a4092bb6e635de3f9a4b7e2a\": container with ID starting with 9a0c571154fa3bd595fc174da831de73d1cf25e9a4092bb6e635de3f9a4b7e2a not found: ID does not exist" Jul 23 13:55:09.079175 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:09.079139 2531 scope.go:117] "RemoveContainer" containerID="2c2ea9d1f4f6f628632f4ce079b870c87282805bfa97e6759563b0aac1ca9804" Jul 23 13:55:09.079407 ip-10-0-130-1 kubenswrapper[2531]: E0723 13:55:09.079390 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c2ea9d1f4f6f628632f4ce079b870c87282805bfa97e6759563b0aac1ca9804\": container with ID starting with 2c2ea9d1f4f6f628632f4ce079b870c87282805bfa97e6759563b0aac1ca9804 not found: ID does not exist" containerID="2c2ea9d1f4f6f628632f4ce079b870c87282805bfa97e6759563b0aac1ca9804" Jul 23 13:55:09.079453 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:09.079416 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c2ea9d1f4f6f628632f4ce079b870c87282805bfa97e6759563b0aac1ca9804"} err="failed to get container status \"2c2ea9d1f4f6f628632f4ce079b870c87282805bfa97e6759563b0aac1ca9804\": rpc error: code = NotFound desc = could not find container \"2c2ea9d1f4f6f628632f4ce079b870c87282805bfa97e6759563b0aac1ca9804\": container with ID starting with 2c2ea9d1f4f6f628632f4ce079b870c87282805bfa97e6759563b0aac1ca9804 not found: ID does not exist" Jul 23 13:55:09.082491 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:09.082470 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p"] Jul 23 13:55:09.086734 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:09.086714 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-690897-openshift-default-57746579f5-kbr5p"] Jul 23 13:55:09.332177 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:09.332098 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81b9e02f-ad6a-4264-b7f9-06558ab5db66" path="/var/lib/kubelet/pods/81b9e02f-ad6a-4264-b7f9-06558ab5db66/volumes" Jul 23 13:55:09.416995 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:09.416956 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-x2pgk_6d067d22-ad1c-414c-8be0-74d3a393b34e/manager/0.log" Jul 23 13:55:09.823633 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:09.823608 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-8gswn_90d7ef40-9d2d-4dcf-902f-53f243bec2d1/server/0.log" Jul 23 13:55:09.974492 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:09.974454 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-fmspk_c2c9af72-c32a-4bba-b76d-d31c9614a6da/manager/0.log" Jul 23 13:55:10.384570 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:10.384542 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-779cfbfdf-tb82k_0271e73f-acbc-4888-b98b-6d3883738a3f/copy-manifests/0.log" Jul 23 13:55:10.395500 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:10.395475 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-779cfbfdf-tb82k_0271e73f-acbc-4888-b98b-6d3883738a3f/manager/0.log" Jul 23 13:55:10.881684 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:10.881657 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29746890-hccpj_fb8f15f4-0f02-447e-94f9-0c8c709b59e7/cleanup/0.log" Jul 23 13:55:12.967482 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:12.967440 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782_0da28128-6818-40bd-92a0-9d8d34b6d9e9/util/0.log" Jul 23 13:55:12.974449 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:12.974418 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782_0da28128-6818-40bd-92a0-9d8d34b6d9e9/pull/0.log" Jul 23 13:55:12.982902 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:12.982884 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782_0da28128-6818-40bd-92a0-9d8d34b6d9e9/extract/0.log" Jul 23 13:55:13.097663 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:13.097630 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr_6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024/util/0.log" Jul 23 13:55:13.104900 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:13.104871 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr_6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024/pull/0.log" Jul 23 13:55:13.112388 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:13.112343 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr_6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024/extract/0.log" Jul 23 13:55:13.230019 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:13.229943 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq_b2bae3f4-18e4-4201-a547-e6850c782cea/util/0.log" Jul 23 13:55:13.236318 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:13.236294 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq_b2bae3f4-18e4-4201-a547-e6850c782cea/pull/0.log" Jul 23 13:55:13.242721 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:13.242699 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq_b2bae3f4-18e4-4201-a547-e6850c782cea/extract/0.log" Jul 23 13:55:13.362527 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:13.362494 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7_c0547351-e583-46d8-a4ba-cf61a7b72075/util/0.log" Jul 23 13:55:13.370210 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:13.370179 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7_c0547351-e583-46d8-a4ba-cf61a7b72075/pull/0.log" Jul 23 13:55:13.377344 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:13.377309 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7_c0547351-e583-46d8-a4ba-cf61a7b72075/extract/0.log" Jul 23 13:55:13.967366 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:13.967331 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-b2kkv_9bdec0ad-3a77-4067-95ca-d13d7a3724ae/registry-server/0.log" Jul 23 13:55:14.221443 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:14.221343 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-v4xtb_50997322-eb3b-4098-ad0a-f9856d8b2c64/limitador/0.log" Jul 23 13:55:14.348377 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:14.348323 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-dkp6n_0ad71b4e-4c7f-46ae-a26f-1d461c3cfd08/manager/0.log" Jul 23 13:55:14.714723 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:14.714695 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678rglnx_56168794-d92a-4663-91dc-babcfc678a53/istio-proxy/0.log" Jul 23 13:55:14.844493 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:14.844453 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-x4whm_72f97835-f1c9-4b7d-9af5-9ccc616b7cd7/discovery/0.log" Jul 23 13:55:15.087655 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:15.087627 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69f7f7699f-rrm9m_f31a5df6-3f63-4aaa-b72e-23628e309ca8/kube-auth-proxy/0.log" Jul 23 13:55:15.214752 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:15.214722 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-lxllp_ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2/istio-proxy/1.log" Jul 23 13:55:16.060574 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:16.060544 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-w9fw5_93ec1d6f-d14a-4df2-87eb-8bc69564a434/main/0.log" Jul 23 13:55:21.896248 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:21.896205 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gps8k/must-gather-7bjtx"] Jul 23 13:55:21.896709 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:21.896693 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0a96c7c-b710-4efb-97c7-6f8e3b96811d" containerName="istio-proxy" Jul 23 13:55:21.896756 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:21.896712 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0a96c7c-b710-4efb-97c7-6f8e3b96811d" containerName="istio-proxy" Jul 23 13:55:21.896756 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:21.896722 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5f9167f-50fb-41c8-bf8d-e2781ee44633" containerName="test-curl-7241-139755629743536" Jul 23 13:55:21.896756 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:21.896728 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5f9167f-50fb-41c8-bf8d-e2781ee44633" containerName="test-curl-7241-139755629743536" Jul 23 13:55:21.896756 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:21.896743 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81b9e02f-ad6a-4264-b7f9-06558ab5db66" containerName="istio-proxy" Jul 23 13:55:21.896756 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:21.896749 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="81b9e02f-ad6a-4264-b7f9-06558ab5db66" containerName="istio-proxy" Jul 23 13:55:21.896942 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:21.896758 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81b9e02f-ad6a-4264-b7f9-06558ab5db66" containerName="istio-proxy" Jul 23 13:55:21.896942 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:21.896766 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="81b9e02f-ad6a-4264-b7f9-06558ab5db66" containerName="istio-proxy" Jul 23 13:55:21.896942 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:21.896844 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0a96c7c-b710-4efb-97c7-6f8e3b96811d" containerName="istio-proxy" Jul 23 13:55:21.896942 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:21.896853 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="81b9e02f-ad6a-4264-b7f9-06558ab5db66" containerName="istio-proxy" Jul 23 13:55:21.896942 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:21.896863 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5f9167f-50fb-41c8-bf8d-e2781ee44633" containerName="test-curl-7241-139755629743536" Jul 23 13:55:21.896942 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:21.896939 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0a96c7c-b710-4efb-97c7-6f8e3b96811d" containerName="istio-proxy" Jul 23 13:55:21.896942 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:21.896945 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0a96c7c-b710-4efb-97c7-6f8e3b96811d" containerName="istio-proxy" Jul 23 13:55:21.897339 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:21.897055 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0a96c7c-b710-4efb-97c7-6f8e3b96811d" containerName="istio-proxy" Jul 23 13:55:21.897339 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:21.897072 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="81b9e02f-ad6a-4264-b7f9-06558ab5db66" containerName="istio-proxy" Jul 23 13:55:21.900005 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:21.899990 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gps8k/must-gather-7bjtx" Jul 23 13:55:21.902460 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:21.902437 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gps8k\"/\"default-dockercfg-6wd9n\"" Jul 23 13:55:21.902606 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:21.902591 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gps8k\"/\"openshift-service-ca.crt\"" Jul 23 13:55:21.903369 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:21.903341 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gps8k\"/\"kube-root-ca.crt\"" Jul 23 13:55:21.910180 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:21.910148 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gps8k/must-gather-7bjtx"] Jul 23 13:55:21.993974 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:21.993938 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jkk6\" (UniqueName: \"kubernetes.io/projected/73eec865-115f-450e-a226-0f8375fd7a79-kube-api-access-2jkk6\") pod \"must-gather-7bjtx\" (UID: \"73eec865-115f-450e-a226-0f8375fd7a79\") " pod="openshift-must-gather-gps8k/must-gather-7bjtx" Jul 23 13:55:21.994137 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:21.993985 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/73eec865-115f-450e-a226-0f8375fd7a79-must-gather-output\") pod \"must-gather-7bjtx\" (UID: \"73eec865-115f-450e-a226-0f8375fd7a79\") " pod="openshift-must-gather-gps8k/must-gather-7bjtx" Jul 23 13:55:22.095377 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:22.095329 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jkk6\" (UniqueName: \"kubernetes.io/projected/73eec865-115f-450e-a226-0f8375fd7a79-kube-api-access-2jkk6\") pod \"must-gather-7bjtx\" (UID: \"73eec865-115f-450e-a226-0f8375fd7a79\") " pod="openshift-must-gather-gps8k/must-gather-7bjtx" Jul 23 13:55:22.095545 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:22.095396 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/73eec865-115f-450e-a226-0f8375fd7a79-must-gather-output\") pod \"must-gather-7bjtx\" (UID: \"73eec865-115f-450e-a226-0f8375fd7a79\") " pod="openshift-must-gather-gps8k/must-gather-7bjtx" Jul 23 13:55:22.095684 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:22.095666 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/73eec865-115f-450e-a226-0f8375fd7a79-must-gather-output\") pod \"must-gather-7bjtx\" (UID: \"73eec865-115f-450e-a226-0f8375fd7a79\") " pod="openshift-must-gather-gps8k/must-gather-7bjtx" Jul 23 13:55:22.104197 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:22.104164 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jkk6\" (UniqueName: \"kubernetes.io/projected/73eec865-115f-450e-a226-0f8375fd7a79-kube-api-access-2jkk6\") pod \"must-gather-7bjtx\" (UID: \"73eec865-115f-450e-a226-0f8375fd7a79\") " pod="openshift-must-gather-gps8k/must-gather-7bjtx" Jul 23 13:55:22.209287 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:22.209206 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gps8k/must-gather-7bjtx" Jul 23 13:55:22.343232 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:22.343207 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gps8k/must-gather-7bjtx"] Jul 23 13:55:22.344198 ip-10-0-130-1 kubenswrapper[2531]: W0723 13:55:22.344173 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73eec865_115f_450e_a226_0f8375fd7a79.slice/crio-d0a4e2dc3d4691fe1a1e34aa5ffdd1e733619ea5d1251b5fd37ee97cd6ed6ea4 WatchSource:0}: Error finding container d0a4e2dc3d4691fe1a1e34aa5ffdd1e733619ea5d1251b5fd37ee97cd6ed6ea4: Status 404 returned error can't find the container with id d0a4e2dc3d4691fe1a1e34aa5ffdd1e733619ea5d1251b5fd37ee97cd6ed6ea4 Jul 23 13:55:23.138232 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:23.138200 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gps8k/must-gather-7bjtx" event={"ID":"73eec865-115f-450e-a226-0f8375fd7a79","Type":"ContainerStarted","Data":"d0a4e2dc3d4691fe1a1e34aa5ffdd1e733619ea5d1251b5fd37ee97cd6ed6ea4"} Jul 23 13:55:24.153093 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:24.152989 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gps8k/must-gather-7bjtx" event={"ID":"73eec865-115f-450e-a226-0f8375fd7a79","Type":"ContainerStarted","Data":"72795aa91680633a9a3b4d1c6903cacadf0cea387de8b5fb249a992a9635bede"} Jul 23 13:55:24.153093 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:24.153073 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gps8k/must-gather-7bjtx" event={"ID":"73eec865-115f-450e-a226-0f8375fd7a79","Type":"ContainerStarted","Data":"fd83edaf0e9aee2dc3a0f13f028d9e9bd91bc4dc762c0487fda9b415a07fefde"} Jul 23 13:55:24.173162 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:24.173104 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gps8k/must-gather-7bjtx" podStartSLOduration=1.7036063000000001 podStartE2EDuration="3.173082924s" totalImagesPullingTime="1.469476624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:55:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:55:24.170259506 +0000 UTC m=+2861.409535485" watchObservedRunningTime="2026-07-23 13:55:24.173082924 +0000 UTC m=+2861.412358902" Jul 23 13:55:25.399639 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:25.399591 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vfgv8_5df5e8b8-2bc3-4788-bf77-4f32a3d52f65/global-pull-secret-syncer/0.log" Jul 23 13:55:25.532559 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:25.532518 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ljjj9_b5ef6285-fd8c-4841-bede-d88d25e6ba2b/konnectivity-agent/0.log" Jul 23 13:55:25.609029 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:25.608996 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-1.ec2.internal_4b3cd48ad004afdc21879d1579b5d29b/haproxy/0.log" Jul 23 13:55:29.426311 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:29.426276 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782_0da28128-6818-40bd-92a0-9d8d34b6d9e9/extract/0.log" Jul 23 13:55:29.454168 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:29.454138 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782_0da28128-6818-40bd-92a0-9d8d34b6d9e9/util/0.log" Jul 23 13:55:29.492905 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:29.492850 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594v782_0da28128-6818-40bd-92a0-9d8d34b6d9e9/pull/0.log" Jul 23 13:55:29.534549 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:29.534512 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr_6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024/extract/0.log" Jul 23 13:55:29.564824 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:29.564790 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr_6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024/util/0.log" Jul 23 13:55:29.592921 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:29.592873 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e045ggr_6d9cc4d4-f4b9-4ac9-84f0-d7eedc1c4024/pull/0.log" Jul 23 13:55:29.630004 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:29.629960 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq_b2bae3f4-18e4-4201-a547-e6850c782cea/extract/0.log" Jul 23 13:55:29.655603 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:29.655569 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq_b2bae3f4-18e4-4201-a547-e6850c782cea/util/0.log" Jul 23 13:55:29.686058 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:29.685980 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bbwbq_b2bae3f4-18e4-4201-a547-e6850c782cea/pull/0.log" Jul 23 13:55:29.723886 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:29.723853 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7_c0547351-e583-46d8-a4ba-cf61a7b72075/extract/0.log" Jul 23 13:55:29.750829 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:29.750802 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7_c0547351-e583-46d8-a4ba-cf61a7b72075/util/0.log" Jul 23 13:55:29.780654 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:29.780623 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rspw7_c0547351-e583-46d8-a4ba-cf61a7b72075/pull/0.log" Jul 23 13:55:30.291008 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:30.290973 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-b2kkv_9bdec0ad-3a77-4067-95ca-d13d7a3724ae/registry-server/0.log" Jul 23 13:55:30.465424 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:30.465394 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-v4xtb_50997322-eb3b-4098-ad0a-f9856d8b2c64/limitador/0.log" Jul 23 13:55:30.554061 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:30.553959 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-dkp6n_0ad71b4e-4c7f-46ae-a26f-1d461c3cfd08/manager/0.log" Jul 23 13:55:32.175095 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:32.175058 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1/alertmanager/0.log" Jul 23 13:55:32.199428 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:32.199401 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1/config-reloader/0.log" Jul 23 13:55:32.223566 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:32.223538 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1/kube-rbac-proxy-web/0.log" Jul 23 13:55:32.247309 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:32.247280 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1/kube-rbac-proxy/0.log" Jul 23 13:55:32.272655 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:32.272615 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1/kube-rbac-proxy-metric/0.log" Jul 23 13:55:32.299691 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:32.299660 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1/prom-label-proxy/0.log" Jul 23 13:55:32.324558 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:32.324511 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5670b455-1fa3-4e4c-8ff1-0ff3ec48a7c1/init-config-reloader/0.log" Jul 23 13:55:32.411721 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:32.411687 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-mfg7x_ce140f37-9913-4f41-8522-377c123c7d65/kube-state-metrics/0.log" Jul 23 13:55:32.445986 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:32.445880 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-mfg7x_ce140f37-9913-4f41-8522-377c123c7d65/kube-rbac-proxy-main/0.log" Jul 23 13:55:32.478221 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:32.478186 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-mfg7x_ce140f37-9913-4f41-8522-377c123c7d65/kube-rbac-proxy-self/0.log" Jul 23 13:55:32.552408 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:32.552373 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-fq7mk_0d5e4e55-0fd8-4802-bd5a-a970f39b27ab/monitoring-plugin/0.log" Jul 23 13:55:32.584463 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:32.584433 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cbk6z_aff84e73-559f-4d6f-a690-c38b353cf763/node-exporter/0.log" Jul 23 13:55:32.629799 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:32.629767 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cbk6z_aff84e73-559f-4d6f-a690-c38b353cf763/kube-rbac-proxy/0.log" Jul 23 13:55:32.657629 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:32.657590 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cbk6z_aff84e73-559f-4d6f-a690-c38b353cf763/init-textfile/0.log" Jul 23 13:55:33.268262 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:33.268223 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-b786d8cbb-vc7wp_ace5549c-7111-4c03-8b91-a0f5c4217529/telemeter-client/0.log" Jul 23 13:55:33.294248 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:33.294210 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-b786d8cbb-vc7wp_ace5549c-7111-4c03-8b91-a0f5c4217529/reload/0.log" Jul 23 13:55:33.316514 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:33.316486 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-b786d8cbb-vc7wp_ace5549c-7111-4c03-8b91-a0f5c4217529/kube-rbac-proxy/0.log" Jul 23 13:55:33.354540 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:33.354514 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8476f75c4-9vzd9_1992ea7e-ea92-4658-8598-3e5b316f69ff/thanos-query/0.log" Jul 23 13:55:33.378732 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:33.378706 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8476f75c4-9vzd9_1992ea7e-ea92-4658-8598-3e5b316f69ff/kube-rbac-proxy-web/0.log" Jul 23 13:55:33.402223 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:33.402148 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8476f75c4-9vzd9_1992ea7e-ea92-4658-8598-3e5b316f69ff/kube-rbac-proxy/0.log" Jul 23 13:55:33.429111 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:33.428978 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8476f75c4-9vzd9_1992ea7e-ea92-4658-8598-3e5b316f69ff/prom-label-proxy/0.log" Jul 23 13:55:33.452810 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:33.452743 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8476f75c4-9vzd9_1992ea7e-ea92-4658-8598-3e5b316f69ff/kube-rbac-proxy-rules/0.log" Jul 23 13:55:33.476631 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:33.476604 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8476f75c4-9vzd9_1992ea7e-ea92-4658-8598-3e5b316f69ff/kube-rbac-proxy-metrics/0.log" Jul 23 13:55:35.801734 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:35.801692 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gps8k/perf-node-gather-daemonset-pzwvz"] Jul 23 13:55:35.807685 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:35.807652 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gps8k/perf-node-gather-daemonset-pzwvz" Jul 23 13:55:35.819249 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:35.819220 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gps8k/perf-node-gather-daemonset-pzwvz"] Jul 23 13:55:35.956514 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:35.956478 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b322d4fd-0502-4fdd-ba87-f0f20a676f7d-proc\") pod \"perf-node-gather-daemonset-pzwvz\" (UID: \"b322d4fd-0502-4fdd-ba87-f0f20a676f7d\") " pod="openshift-must-gather-gps8k/perf-node-gather-daemonset-pzwvz" Jul 23 13:55:35.956708 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:35.956525 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b322d4fd-0502-4fdd-ba87-f0f20a676f7d-lib-modules\") pod \"perf-node-gather-daemonset-pzwvz\" (UID: \"b322d4fd-0502-4fdd-ba87-f0f20a676f7d\") " pod="openshift-must-gather-gps8k/perf-node-gather-daemonset-pzwvz" Jul 23 13:55:35.956708 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:35.956553 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c268t\" (UniqueName: \"kubernetes.io/projected/b322d4fd-0502-4fdd-ba87-f0f20a676f7d-kube-api-access-c268t\") pod \"perf-node-gather-daemonset-pzwvz\" (UID: \"b322d4fd-0502-4fdd-ba87-f0f20a676f7d\") " pod="openshift-must-gather-gps8k/perf-node-gather-daemonset-pzwvz" Jul 23 13:55:35.956708 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:35.956608 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b322d4fd-0502-4fdd-ba87-f0f20a676f7d-sys\") pod \"perf-node-gather-daemonset-pzwvz\" (UID: \"b322d4fd-0502-4fdd-ba87-f0f20a676f7d\") " pod="openshift-must-gather-gps8k/perf-node-gather-daemonset-pzwvz" Jul 23 13:55:35.956708 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:35.956686 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b322d4fd-0502-4fdd-ba87-f0f20a676f7d-podres\") pod \"perf-node-gather-daemonset-pzwvz\" (UID: \"b322d4fd-0502-4fdd-ba87-f0f20a676f7d\") " pod="openshift-must-gather-gps8k/perf-node-gather-daemonset-pzwvz" Jul 23 13:55:35.994736 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:35.994686 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-96945b589-8n98g_62544614-8aad-4ff5-9908-4a58860c5f75/console/0.log" Jul 23 13:55:36.057992 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:36.057958 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b322d4fd-0502-4fdd-ba87-f0f20a676f7d-proc\") pod \"perf-node-gather-daemonset-pzwvz\" (UID: \"b322d4fd-0502-4fdd-ba87-f0f20a676f7d\") " pod="openshift-must-gather-gps8k/perf-node-gather-daemonset-pzwvz" Jul 23 13:55:36.058167 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:36.058005 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b322d4fd-0502-4fdd-ba87-f0f20a676f7d-lib-modules\") pod \"perf-node-gather-daemonset-pzwvz\" (UID: \"b322d4fd-0502-4fdd-ba87-f0f20a676f7d\") " pod="openshift-must-gather-gps8k/perf-node-gather-daemonset-pzwvz" Jul 23 13:55:36.058167 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:36.058041 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c268t\" (UniqueName: \"kubernetes.io/projected/b322d4fd-0502-4fdd-ba87-f0f20a676f7d-kube-api-access-c268t\") pod \"perf-node-gather-daemonset-pzwvz\" (UID: \"b322d4fd-0502-4fdd-ba87-f0f20a676f7d\") " pod="openshift-must-gather-gps8k/perf-node-gather-daemonset-pzwvz" Jul 23 13:55:36.058167 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:36.058075 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b322d4fd-0502-4fdd-ba87-f0f20a676f7d-sys\") pod \"perf-node-gather-daemonset-pzwvz\" (UID: \"b322d4fd-0502-4fdd-ba87-f0f20a676f7d\") " pod="openshift-must-gather-gps8k/perf-node-gather-daemonset-pzwvz" Jul 23 13:55:36.058167 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:36.058085 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b322d4fd-0502-4fdd-ba87-f0f20a676f7d-proc\") pod \"perf-node-gather-daemonset-pzwvz\" (UID: \"b322d4fd-0502-4fdd-ba87-f0f20a676f7d\") " pod="openshift-must-gather-gps8k/perf-node-gather-daemonset-pzwvz" Jul 23 13:55:36.058313 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:36.058217 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b322d4fd-0502-4fdd-ba87-f0f20a676f7d-lib-modules\") pod \"perf-node-gather-daemonset-pzwvz\" (UID: \"b322d4fd-0502-4fdd-ba87-f0f20a676f7d\") " pod="openshift-must-gather-gps8k/perf-node-gather-daemonset-pzwvz" Jul 23 13:55:36.058313 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:36.058260 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b322d4fd-0502-4fdd-ba87-f0f20a676f7d-sys\") pod \"perf-node-gather-daemonset-pzwvz\" (UID: \"b322d4fd-0502-4fdd-ba87-f0f20a676f7d\") " pod="openshift-must-gather-gps8k/perf-node-gather-daemonset-pzwvz" Jul 23 13:55:36.058422 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:36.058320 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b322d4fd-0502-4fdd-ba87-f0f20a676f7d-podres\") pod \"perf-node-gather-daemonset-pzwvz\" (UID: \"b322d4fd-0502-4fdd-ba87-f0f20a676f7d\") " pod="openshift-must-gather-gps8k/perf-node-gather-daemonset-pzwvz" Jul 23 13:55:36.058571 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:36.058547 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b322d4fd-0502-4fdd-ba87-f0f20a676f7d-podres\") pod \"perf-node-gather-daemonset-pzwvz\" (UID: \"b322d4fd-0502-4fdd-ba87-f0f20a676f7d\") " pod="openshift-must-gather-gps8k/perf-node-gather-daemonset-pzwvz" Jul 23 13:55:36.070903 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:36.070835 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c268t\" (UniqueName: \"kubernetes.io/projected/b322d4fd-0502-4fdd-ba87-f0f20a676f7d-kube-api-access-c268t\") pod \"perf-node-gather-daemonset-pzwvz\" (UID: \"b322d4fd-0502-4fdd-ba87-f0f20a676f7d\") " pod="openshift-must-gather-gps8k/perf-node-gather-daemonset-pzwvz" Jul 23 13:55:36.123931 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:36.123888 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gps8k/perf-node-gather-daemonset-pzwvz" Jul 23 13:55:36.313934 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:36.313814 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gps8k/perf-node-gather-daemonset-pzwvz"] Jul 23 13:55:37.262945 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:37.262900 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gps8k/perf-node-gather-daemonset-pzwvz" event={"ID":"b322d4fd-0502-4fdd-ba87-f0f20a676f7d","Type":"ContainerStarted","Data":"b2ebcf882d6bb35154349ced6dca2aefc9146c89329290dfef7f4257e41763b2"} Jul 23 13:55:37.263454 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:37.263007 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gps8k/perf-node-gather-daemonset-pzwvz" event={"ID":"b322d4fd-0502-4fdd-ba87-f0f20a676f7d","Type":"ContainerStarted","Data":"c129760c2d54de56eff0984e1b552436024158a66b9dd42e20ff672d6b740cf3"} Jul 23 13:55:37.263454 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:37.263257 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gps8k/perf-node-gather-daemonset-pzwvz" Jul 23 13:55:37.772822 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:37.772790 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5zfnk_b50baffc-9fe0-4ba8-9292-ca620a946b89/dns/0.log" Jul 23 13:55:37.794656 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:37.794625 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5zfnk_b50baffc-9fe0-4ba8-9292-ca620a946b89/kube-rbac-proxy/0.log" Jul 23 13:55:37.957336 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:37.957309 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xlb6j_7af68660-8ebf-43b5-981e-ec1159a7ca1a/dns-node-resolver/0.log" Jul 23 13:55:38.503124 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:38.503091 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7d4fcff47f-52ds5_459f4b73-abfd-426c-abcd-7de9467a3fab/registry/0.log" Jul 23 13:55:38.558067 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:38.558029 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-f9q95_007b5917-9a4d-45b8-8ce2-e1639c8db863/node-ca/0.log" Jul 23 13:55:39.517909 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:39.516764 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678rglnx_56168794-d92a-4663-91dc-babcfc678a53/istio-proxy/0.log" Jul 23 13:55:39.756707 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:39.756673 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-x4whm_72f97835-f1c9-4b7d-9af5-9ccc616b7cd7/discovery/0.log" Jul 23 13:55:39.812455 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:39.812428 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69f7f7699f-rrm9m_f31a5df6-3f63-4aaa-b72e-23628e309ca8/kube-auth-proxy/0.log" Jul 23 13:55:39.843235 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:39.843192 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-lxllp_ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2/istio-proxy/0.log" Jul 23 13:55:39.898089 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:39.898056 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-lxllp_ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2/istio-proxy/1.log" Jul 23 13:55:40.735744 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:40.735703 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cd2jt_8261ecc7-323a-4dd7-8a6a-43381de92427/serve-healthcheck-canary/0.log" Jul 23 13:55:41.316972 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:41.316940 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cqn57_e39a3e4d-f59e-4de3-b64d-3924df816483/kube-rbac-proxy/0.log" Jul 23 13:55:41.343321 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:41.343292 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cqn57_e39a3e4d-f59e-4de3-b64d-3924df816483/exporter/0.log" Jul 23 13:55:41.368190 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:41.368159 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cqn57_e39a3e4d-f59e-4de3-b64d-3924df816483/extractor/0.log" Jul 23 13:55:43.284615 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:43.284580 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gps8k/perf-node-gather-daemonset-pzwvz" Jul 23 13:55:43.306348 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:43.306289 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gps8k/perf-node-gather-daemonset-pzwvz" podStartSLOduration=8.306273569 podStartE2EDuration="8.306273569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:55:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:55:37.280338847 +0000 UTC m=+2874.519614823" watchObservedRunningTime="2026-07-23 13:55:43.306273569 +0000 UTC m=+2880.545549546" Jul 23 13:55:43.551200 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:43.551097 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-x2pgk_6d067d22-ad1c-414c-8be0-74d3a393b34e/manager/0.log" Jul 23 13:55:43.870782 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:43.870750 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-8gswn_90d7ef40-9d2d-4dcf-902f-53f243bec2d1/server/0.log" Jul 23 13:55:43.986421 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:43.986381 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-fmspk_c2c9af72-c32a-4bba-b76d-d31c9614a6da/manager/0.log" Jul 23 13:55:44.163078 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:44.162990 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-779cfbfdf-tb82k_0271e73f-acbc-4888-b98b-6d3883738a3f/manager/0.log" Jul 23 13:55:44.184700 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:44.184673 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-779cfbfdf-tb82k_0271e73f-acbc-4888-b98b-6d3883738a3f/copy-manifests/0.log" Jul 23 13:55:46.200039 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:46.200003 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678rglnx_56168794-d92a-4663-91dc-babcfc678a53/istio-proxy/0.log" Jul 23 13:55:46.440460 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:46.440379 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-x4whm_72f97835-f1c9-4b7d-9af5-9ccc616b7cd7/discovery/0.log" Jul 23 13:55:46.477859 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:46.477833 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69f7f7699f-rrm9m_f31a5df6-3f63-4aaa-b72e-23628e309ca8/kube-auth-proxy/0.log" Jul 23 13:55:46.507850 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:46.507815 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-lxllp_ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2/istio-proxy/0.log" Jul 23 13:55:46.564377 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:46.564325 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-lxllp_ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2/istio-proxy/1.log" Jul 23 13:55:47.841198 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:47.841150 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678rglnx_56168794-d92a-4663-91dc-babcfc678a53/istio-proxy/0.log" Jul 23 13:55:48.043305 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:48.043222 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-x4whm_72f97835-f1c9-4b7d-9af5-9ccc616b7cd7/discovery/0.log" Jul 23 13:55:48.076607 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:48.076576 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69f7f7699f-rrm9m_f31a5df6-3f63-4aaa-b72e-23628e309ca8/kube-auth-proxy/0.log" Jul 23 13:55:48.104571 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:48.104538 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-lxllp_ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2/istio-proxy/0.log" Jul 23 13:55:48.150832 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:48.150285 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-lxllp_ff43aeb7-d1b7-425e-94fd-cf95f4e44ed2/istio-proxy/1.log" Jul 23 13:55:52.721625 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:52.721597 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwp6q_e6fae90e-8282-40c3-9720-4136984b837a/kube-multus-additional-cni-plugins/0.log" Jul 23 13:55:52.742728 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:52.742703 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwp6q_e6fae90e-8282-40c3-9720-4136984b837a/egress-router-binary-copy/0.log" Jul 23 13:55:52.764600 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:52.764571 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwp6q_e6fae90e-8282-40c3-9720-4136984b837a/cni-plugins/0.log" Jul 23 13:55:52.789146 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:52.789120 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwp6q_e6fae90e-8282-40c3-9720-4136984b837a/bond-cni-plugin/0.log" Jul 23 13:55:52.812227 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:52.812196 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwp6q_e6fae90e-8282-40c3-9720-4136984b837a/routeoverride-cni/0.log" Jul 23 13:55:52.833217 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:52.833193 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwp6q_e6fae90e-8282-40c3-9720-4136984b837a/whereabouts-cni-bincopy/0.log" Jul 23 13:55:52.854926 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:52.854894 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwp6q_e6fae90e-8282-40c3-9720-4136984b837a/whereabouts-cni/0.log" Jul 23 13:55:52.933761 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:52.933715 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-gckpb_5839c342-f734-4f2e-8093-398fe2d82ed9/kube-multus/0.log" Jul 23 13:55:53.030183 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:53.030076 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cp74b_e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3/network-metrics-daemon/0.log" Jul 23 13:55:53.063821 ip-10-0-130-1 kubenswrapper[2531]: I0723 13:55:53.063787 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cp74b_e0ceb6d6-8b7c-4e99-9d9e-16c3493425a3/kube-rbac-proxy/0.log"