Jul 09 13:27:37.094090 ip-10-0-142-87 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 09 13:27:37.094100 ip-10-0-142-87 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 09 13:27:37.094107 ip-10-0-142-87 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 09 13:27:37.094265 ip-10-0-142-87 systemd[1]: Failed to start Kubernetes Kubelet. Jul 09 13:27:47.280570 ip-10-0-142-87 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 09 13:27:47.280589 ip-10-0-142-87 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 09b80f06b11c4b57be8021b4615ba27b -- Jul 09 13:29:41.227126 ip-10-0-142-87 systemd[1]: Starting Kubernetes Kubelet... Jul 09 13:29:41.575874 ip-10-0-142-87 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 13:29:41.575874 ip-10-0-142-87 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 13:29:41.575874 ip-10-0-142-87 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 13:29:41.575874 ip-10-0-142-87 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 13:29:41.575874 ip-10-0-142-87 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 13:29:41.577507 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.577358 2573 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 13:29:41.583857 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583828 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 13:29:41.583857 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583851 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 13:29:41.583857 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583855 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 13:29:41.583857 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583859 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 13:29:41.583857 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583861 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 13:29:41.583857 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583865 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 13:29:41.583857 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583868 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 13:29:41.584135 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583871 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 13:29:41.584135 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583874 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 13:29:41.584135 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583877 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 13:29:41.584135 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583894 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 13:29:41.584135 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583897 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 13:29:41.584135 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583900 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 13:29:41.584135 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583903 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 13:29:41.584135 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583906 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 13:29:41.584135 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583909 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 13:29:41.584135 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583912 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 13:29:41.584135 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583914 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 13:29:41.584135 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583918 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 13:29:41.584135 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583921 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 13:29:41.584135 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583924 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 13:29:41.584135 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583926 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 13:29:41.584135 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583929 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 13:29:41.584135 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583931 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 13:29:41.584135 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583934 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 13:29:41.584135 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583937 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 13:29:41.584135 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583939 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 13:29:41.584595 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583942 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 13:29:41.584595 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583944 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 13:29:41.584595 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583947 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 13:29:41.584595 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583957 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 13:29:41.584595 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583959 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 13:29:41.584595 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583962 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 09 13:29:41.584595 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583965 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 13:29:41.584595 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583967 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 13:29:41.584595 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583970 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 13:29:41.584595 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583973 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 13:29:41.584595 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583976 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 13:29:41.584595 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583979 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 13:29:41.584595 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583982 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 13:29:41.584595 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583985 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 13:29:41.584595 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583988 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 13:29:41.584595 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583992 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 13:29:41.584595 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583996 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 13:29:41.584595 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.583999 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 13:29:41.584595 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584002 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 13:29:41.585059 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584005 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 13:29:41.585059 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584007 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 13:29:41.585059 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584010 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 13:29:41.585059 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584013 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 13:29:41.585059 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584016 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 13:29:41.585059 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584019 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 13:29:41.585059 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584021 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 13:29:41.585059 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584024 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 13:29:41.585059 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584026 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 13:29:41.585059 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584029 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 13:29:41.585059 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584034 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 13:29:41.585059 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584037 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 13:29:41.585059 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584041 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 13:29:41.585059 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584044 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 13:29:41.585059 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584048 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 13:29:41.585059 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584051 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 13:29:41.585059 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584054 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 13:29:41.585059 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584057 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 13:29:41.585059 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584061 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 13:29:41.585543 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584064 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 13:29:41.585543 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584067 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 13:29:41.585543 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584070 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 13:29:41.585543 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584072 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 13:29:41.585543 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584075 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 13:29:41.585543 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584078 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 13:29:41.585543 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584080 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 13:29:41.585543 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584083 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 13:29:41.585543 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584086 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 13:29:41.585543 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584088 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 13:29:41.585543 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584091 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 13:29:41.585543 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584093 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 13:29:41.585543 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584096 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 13:29:41.585543 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584099 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 13:29:41.585543 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584101 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 13:29:41.585543 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584104 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 13:29:41.585543 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584107 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 13:29:41.585543 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584109 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 13:29:41.585543 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584112 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 13:29:41.585543 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584115 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 13:29:41.586045 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584117 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 13:29:41.586045 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584120 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 13:29:41.586045 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584123 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 13:29:41.586045 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584126 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 13:29:41.586045 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584128 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 13:29:41.586045 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584130 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 13:29:41.586045 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584133 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 13:29:41.586045 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584135 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 13:29:41.586045 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584138 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 13:29:41.586045 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584141 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 13:29:41.586045 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584144 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 13:29:41.586045 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584147 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 13:29:41.586045 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584150 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 13:29:41.586045 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.584153 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 13:29:41.586045 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.585990 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 13:29:41.586045 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586000 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 13:29:41.586045 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586003 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 13:29:41.586045 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586006 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 13:29:41.586045 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586009 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 13:29:41.586498 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586012 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 13:29:41.586498 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586015 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 13:29:41.586498 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586018 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 13:29:41.586498 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586020 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 13:29:41.586498 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586024 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 13:29:41.586498 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586028 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 13:29:41.586498 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586031 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 13:29:41.586498 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586034 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 13:29:41.586498 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586037 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 13:29:41.586498 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586039 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 13:29:41.586498 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586042 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 13:29:41.586498 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586045 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 13:29:41.586498 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586048 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 13:29:41.586498 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586051 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 13:29:41.586498 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586054 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 13:29:41.586498 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586058 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 13:29:41.586498 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586061 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 13:29:41.586498 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586063 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 13:29:41.586498 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586066 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 13:29:41.586973 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586068 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 13:29:41.586973 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586071 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 13:29:41.586973 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586074 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 13:29:41.586973 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586077 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 13:29:41.586973 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586079 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 13:29:41.586973 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586082 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 13:29:41.586973 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586084 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 13:29:41.586973 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586093 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 13:29:41.586973 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586096 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 13:29:41.586973 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586098 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 13:29:41.586973 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586101 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 13:29:41.586973 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586104 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 13:29:41.586973 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586106 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 13:29:41.586973 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586109 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 13:29:41.586973 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586113 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 13:29:41.586973 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586116 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 13:29:41.586973 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586119 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 13:29:41.586973 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586121 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 13:29:41.586973 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586124 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 13:29:41.586973 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586127 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 13:29:41.587468 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586130 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 13:29:41.587468 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586132 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 13:29:41.587468 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586135 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 13:29:41.587468 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586137 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 09 13:29:41.587468 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586140 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 13:29:41.587468 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586142 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 13:29:41.587468 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586146 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 13:29:41.587468 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586149 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 13:29:41.587468 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586151 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 13:29:41.587468 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586154 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 13:29:41.587468 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586156 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 13:29:41.587468 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586159 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 13:29:41.587468 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586161 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 13:29:41.587468 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586165 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 13:29:41.587468 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586169 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 13:29:41.587468 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586171 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 13:29:41.587468 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586175 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 13:29:41.587468 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586177 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 13:29:41.587468 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586180 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 13:29:41.587933 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586183 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 13:29:41.587933 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586185 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 13:29:41.587933 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586188 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 13:29:41.587933 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586191 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 13:29:41.587933 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586193 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 13:29:41.587933 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586196 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 13:29:41.587933 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586200 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 13:29:41.587933 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586203 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 13:29:41.587933 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586206 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 13:29:41.587933 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586208 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 13:29:41.587933 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586211 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 13:29:41.587933 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586214 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 13:29:41.587933 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586217 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 13:29:41.587933 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586219 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 13:29:41.587933 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586222 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 13:29:41.587933 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586224 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 13:29:41.587933 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586227 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 13:29:41.587933 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586229 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 13:29:41.587933 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586232 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 13:29:41.587933 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586234 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 13:29:41.588409 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586237 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 13:29:41.588409 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586240 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 13:29:41.588409 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586243 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 13:29:41.588409 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586246 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 13:29:41.588409 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586249 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 13:29:41.588409 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586252 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 13:29:41.588409 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586254 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 13:29:41.588409 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586256 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 13:29:41.588409 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586260 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 13:29:41.588409 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586263 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 13:29:41.588409 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586266 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 13:29:41.588409 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586269 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 13:29:41.588409 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586271 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 13:29:41.588409 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586274 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 13:29:41.588409 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586276 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 13:29:41.588409 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586279 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 13:29:41.588409 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586669 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 13:29:41.588409 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586674 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 13:29:41.588409 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586676 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 13:29:41.588409 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586679 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 13:29:41.588906 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586682 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 13:29:41.588906 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586685 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 13:29:41.588906 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586688 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 13:29:41.588906 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586690 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 13:29:41.588906 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586693 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 13:29:41.588906 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586696 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 13:29:41.588906 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586698 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 13:29:41.588906 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586701 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 13:29:41.588906 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586703 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 13:29:41.588906 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586706 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 13:29:41.588906 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586708 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 13:29:41.588906 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586711 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 13:29:41.588906 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586716 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 13:29:41.588906 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586719 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 13:29:41.588906 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586723 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 13:29:41.588906 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586727 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 13:29:41.588906 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586730 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 13:29:41.588906 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586734 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 13:29:41.588906 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586738 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 13:29:41.589378 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586741 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 13:29:41.589378 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586744 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 13:29:41.589378 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586747 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 13:29:41.589378 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586749 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 13:29:41.589378 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586752 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 13:29:41.589378 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586754 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 13:29:41.589378 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586757 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 13:29:41.589378 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586759 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 13:29:41.589378 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586762 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 13:29:41.589378 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586764 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 13:29:41.589378 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586768 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 13:29:41.589378 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586770 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 13:29:41.589378 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586773 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 13:29:41.589378 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586776 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 13:29:41.589378 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586779 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 13:29:41.589378 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586781 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 13:29:41.589378 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586784 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 13:29:41.589378 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586787 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 13:29:41.589378 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586789 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 13:29:41.589378 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586792 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 13:29:41.589932 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586794 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 13:29:41.589932 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586797 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 13:29:41.589932 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586799 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 13:29:41.589932 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586802 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 13:29:41.589932 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586804 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 13:29:41.589932 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586807 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 13:29:41.589932 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586810 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 13:29:41.589932 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586812 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 13:29:41.589932 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586816 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 13:29:41.589932 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586818 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 13:29:41.589932 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586821 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 13:29:41.589932 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586823 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 13:29:41.589932 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586826 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 13:29:41.589932 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586828 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 13:29:41.589932 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586830 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 13:29:41.589932 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586833 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 13:29:41.589932 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586835 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 13:29:41.589932 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586838 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 13:29:41.589932 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586840 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 13:29:41.589932 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586843 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 13:29:41.590457 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586846 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 13:29:41.590457 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586848 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 13:29:41.590457 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586856 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 13:29:41.590457 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586858 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 13:29:41.590457 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586863 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 13:29:41.590457 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586866 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 13:29:41.590457 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586869 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 13:29:41.590457 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586871 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 13:29:41.590457 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586874 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 13:29:41.590457 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586876 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 13:29:41.590457 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586879 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 13:29:41.590457 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586895 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 13:29:41.590457 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586898 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 13:29:41.590457 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586901 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 13:29:41.590457 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586903 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 13:29:41.590457 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586906 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 13:29:41.590457 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586909 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 09 13:29:41.590457 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586911 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 13:29:41.590457 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586914 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 13:29:41.590457 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586917 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 13:29:41.590965 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586920 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 13:29:41.590965 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586922 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 13:29:41.590965 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586925 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 13:29:41.590965 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586928 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 13:29:41.590965 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586930 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 13:29:41.590965 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586933 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 13:29:41.590965 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586935 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 13:29:41.590965 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586938 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 13:29:41.590965 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586940 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 13:29:41.590965 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586943 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 13:29:41.590965 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586946 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 13:29:41.590965 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586948 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 13:29:41.590965 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586951 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 13:29:41.590965 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586953 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 13:29:41.590965 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586960 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 13:29:41.590965 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.586963 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 13:29:41.590965 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587039 2573 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 13:29:41.590965 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587046 2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 13:29:41.590965 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587081 2573 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 13:29:41.590965 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587086 2573 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 13:29:41.591483 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587091 2573 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 13:29:41.591483 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587094 2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 13:29:41.591483 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587099 2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 13:29:41.591483 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587103 2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 13:29:41.591483 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587106 2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 13:29:41.591483 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587109 2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 13:29:41.591483 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587113 2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 13:29:41.591483 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587117 2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 13:29:41.591483 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587120 2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 13:29:41.591483 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587122 2573 flags.go:64] FLAG: --cgroup-root="" Jul 09 13:29:41.591483 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587125 2573 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 13:29:41.591483 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587129 2573 flags.go:64] FLAG: --client-ca-file="" Jul 09 13:29:41.591483 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587132 2573 flags.go:64] FLAG: --cloud-provider="external" Jul 09 13:29:41.591483 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587135 2573 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 13:29:41.591483 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587139 2573 flags.go:64] FLAG: --cluster-domain="" Jul 09 13:29:41.591483 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587142 2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 13:29:41.591483 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587145 2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 13:29:41.591483 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587148 2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 13:29:41.591483 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587151 2573 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 13:29:41.591483 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587155 2573 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 13:29:41.591483 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587158 2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 13:29:41.591483 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587161 2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 13:29:41.591483 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587164 2573 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 13:29:41.592092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587167 2573 flags.go:64] FLAG: --contention-profiling="false" Jul 09 13:29:41.592092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587170 2573 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 13:29:41.592092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587173 2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 13:29:41.592092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587177 2573 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 13:29:41.592092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587180 2573 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 13:29:41.592092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587184 2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 13:29:41.592092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587187 2573 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 13:29:41.592092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587190 2573 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 13:29:41.592092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587193 2573 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 13:29:41.592092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587196 2573 flags.go:64] FLAG: --enable-server="true" Jul 09 13:29:41.592092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587199 2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 13:29:41.592092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587204 2573 flags.go:64] FLAG: --event-burst="100" Jul 09 13:29:41.592092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587207 2573 flags.go:64] FLAG: --event-qps="50" Jul 09 13:29:41.592092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587210 2573 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 13:29:41.592092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587213 2573 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 13:29:41.592092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587216 2573 flags.go:64] FLAG: --eviction-hard="" Jul 09 13:29:41.592092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587220 2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 13:29:41.592092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587222 2573 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 13:29:41.592092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587225 2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 13:29:41.592092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587228 2573 flags.go:64] FLAG: --eviction-soft="" Jul 09 13:29:41.592092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587231 2573 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 13:29:41.592092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587233 2573 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 13:29:41.592092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587237 2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 13:29:41.592092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587240 2573 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 13:29:41.592092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587243 2573 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 13:29:41.592747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587246 2573 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 13:29:41.592747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587248 2573 flags.go:64] FLAG: --feature-gates="" Jul 09 13:29:41.592747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587252 2573 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 13:29:41.592747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587255 2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 13:29:41.592747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587258 2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 13:29:41.592747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587261 2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 13:29:41.592747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587264 2573 flags.go:64] FLAG: --healthz-port="10248" Jul 09 13:29:41.592747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587267 2573 flags.go:64] FLAG: --help="false" Jul 09 13:29:41.592747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587270 2573 flags.go:64] FLAG: --hostname-override="ip-10-0-142-87.ec2.internal" Jul 09 13:29:41.592747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587273 2573 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 13:29:41.592747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587277 2573 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 13:29:41.592747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587280 2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 13:29:41.592747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587283 2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 13:29:41.592747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587286 2573 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 13:29:41.592747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587289 2573 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 13:29:41.592747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587292 2573 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 13:29:41.592747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587295 2573 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 13:29:41.592747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587298 2573 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 13:29:41.592747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587301 2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 13:29:41.592747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587305 2573 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 13:29:41.592747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587308 2573 flags.go:64] FLAG: --kube-reserved="" Jul 09 13:29:41.592747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587310 2573 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 13:29:41.592747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587313 2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 13:29:41.592747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587316 2573 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 13:29:41.593348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587319 2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 13:29:41.593348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587322 2573 flags.go:64] FLAG: --lock-file="" Jul 09 13:29:41.593348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587325 2573 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 13:29:41.593348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587328 2573 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 13:29:41.593348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587331 2573 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 13:29:41.593348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587336 2573 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 13:29:41.593348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587339 2573 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 13:29:41.593348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587342 2573 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 13:29:41.593348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587345 2573 flags.go:64] FLAG: --logging-format="text" Jul 09 13:29:41.593348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587348 2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 13:29:41.593348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587351 2573 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 13:29:41.593348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587354 2573 flags.go:64] FLAG: --manifest-url="" Jul 09 13:29:41.593348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587357 2573 flags.go:64] FLAG: --manifest-url-header="" Jul 09 13:29:41.593348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587361 2573 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 13:29:41.593348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587364 2573 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 13:29:41.593348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587368 2573 flags.go:64] FLAG: --max-pods="110" Jul 09 13:29:41.593348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587371 2573 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 13:29:41.593348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587374 2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 13:29:41.593348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587377 2573 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 13:29:41.593348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587380 2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 13:29:41.593348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587383 2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 13:29:41.593348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587386 2573 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 13:29:41.593348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587389 2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 13:29:41.593348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587397 2573 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 13:29:41.593348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587400 2573 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 13:29:41.594026 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587403 2573 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 13:29:41.594026 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587406 2573 flags.go:64] FLAG: --pod-cidr="" Jul 09 13:29:41.594026 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587409 2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 09 13:29:41.594026 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587416 2573 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 13:29:41.594026 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587419 2573 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 13:29:41.594026 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587422 2573 flags.go:64] FLAG: --pods-per-core="0" Jul 09 13:29:41.594026 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587425 2573 flags.go:64] FLAG: --port="10250" Jul 09 13:29:41.594026 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587428 2573 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 13:29:41.594026 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587431 2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02f53ebf8ad03782b" Jul 09 13:29:41.594026 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587434 2573 flags.go:64] FLAG: --qos-reserved="" Jul 09 13:29:41.594026 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587437 2573 flags.go:64] FLAG: --read-only-port="10255" Jul 09 13:29:41.594026 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587440 2573 flags.go:64] FLAG: --register-node="true" Jul 09 13:29:41.594026 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587442 2573 flags.go:64] FLAG: --register-with-taints="" Jul 09 13:29:41.594026 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587446 2573 flags.go:64] FLAG: --registry-burst="10" Jul 09 13:29:41.594026 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587450 2573 flags.go:64] FLAG: --registry-qps="5" Jul 09 13:29:41.594026 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587453 2573 flags.go:64] FLAG: --reserved-cpus="" Jul 09 13:29:41.594026 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587456 2573 flags.go:64] FLAG: --reserved-memory="" Jul 09 13:29:41.594026 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587459 2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 13:29:41.594026 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587462 2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 13:29:41.594026 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587465 2573 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 13:29:41.594026 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587468 2573 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 13:29:41.594026 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587471 2573 flags.go:64] FLAG: --runonce="false" Jul 09 13:29:41.594026 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587474 2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 13:29:41.594026 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587477 2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 13:29:41.594026 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587481 2573 flags.go:64] FLAG: --seccomp-default="false" Jul 09 13:29:41.594613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587484 2573 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 13:29:41.594613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587487 2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 13:29:41.594613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587489 2573 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 13:29:41.594613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587492 2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 13:29:41.594613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587496 2573 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 13:29:41.594613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587499 2573 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 13:29:41.594613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587502 2573 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 13:29:41.594613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587505 2573 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 13:29:41.594613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587508 2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 13:29:41.594613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587511 2573 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 13:29:41.594613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587514 2573 flags.go:64] FLAG: --system-cgroups="" Jul 09 13:29:41.594613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587516 2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 13:29:41.594613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587522 2573 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 13:29:41.594613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587524 2573 flags.go:64] FLAG: --tls-cert-file="" Jul 09 13:29:41.594613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587528 2573 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 13:29:41.594613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587532 2573 flags.go:64] FLAG: --tls-min-version="" Jul 09 13:29:41.594613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587535 2573 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 13:29:41.594613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587537 2573 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 13:29:41.594613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587540 2573 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 13:29:41.594613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587543 2573 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 13:29:41.594613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587545 2573 flags.go:64] FLAG: --v="2" Jul 09 13:29:41.594613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587550 2573 flags.go:64] FLAG: --version="false" Jul 09 13:29:41.594613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587555 2573 flags.go:64] FLAG: --vmodule="" Jul 09 13:29:41.594613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587560 2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 13:29:41.594613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.587563 2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 13:29:41.595233 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587692 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 13:29:41.595233 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587696 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 13:29:41.595233 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587699 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 13:29:41.595233 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587702 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 13:29:41.595233 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587705 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 13:29:41.595233 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587708 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 13:29:41.595233 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587712 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 13:29:41.595233 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587715 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 13:29:41.595233 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587717 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 13:29:41.595233 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587720 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 13:29:41.595233 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587723 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 13:29:41.595233 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587725 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 13:29:41.595233 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587728 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 13:29:41.595233 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587731 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 13:29:41.595233 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587733 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 13:29:41.595233 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587736 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 13:29:41.595233 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587739 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 13:29:41.595233 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587741 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 13:29:41.595233 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587743 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 13:29:41.595233 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587746 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 13:29:41.595713 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587748 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 13:29:41.595713 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587751 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 13:29:41.595713 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587754 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 13:29:41.595713 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587756 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 13:29:41.595713 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587759 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 13:29:41.595713 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587761 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 13:29:41.595713 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587764 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 13:29:41.595713 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587766 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 13:29:41.595713 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587769 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 13:29:41.595713 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587771 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 13:29:41.595713 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587776 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 13:29:41.595713 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587779 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 13:29:41.595713 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587781 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 13:29:41.595713 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587784 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 13:29:41.595713 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587786 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 13:29:41.595713 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587789 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 13:29:41.595713 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587791 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 13:29:41.595713 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587794 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 13:29:41.595713 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587797 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 13:29:41.595713 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587800 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 13:29:41.596264 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587803 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 13:29:41.596264 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587805 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 13:29:41.596264 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587808 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 13:29:41.596264 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587810 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 13:29:41.596264 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587813 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 13:29:41.596264 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587815 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 13:29:41.596264 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587818 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 13:29:41.596264 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587821 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 13:29:41.596264 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587823 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 13:29:41.596264 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587826 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 13:29:41.596264 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587829 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 13:29:41.596264 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587832 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 13:29:41.596264 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587837 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 13:29:41.596264 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587841 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 13:29:41.596264 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587844 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 13:29:41.596264 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587847 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 13:29:41.596264 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587850 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 13:29:41.596264 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587852 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 13:29:41.596264 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587855 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 13:29:41.596724 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587857 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 13:29:41.596724 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587860 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 13:29:41.596724 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587862 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 13:29:41.596724 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587866 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 13:29:41.596724 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587869 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 13:29:41.596724 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587872 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 13:29:41.596724 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587874 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 13:29:41.596724 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587877 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 13:29:41.596724 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587879 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 13:29:41.596724 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587897 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 13:29:41.596724 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587900 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 13:29:41.596724 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587904 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 13:29:41.596724 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587906 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 13:29:41.596724 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587909 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 13:29:41.596724 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587912 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 13:29:41.596724 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587914 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 13:29:41.596724 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587917 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 09 13:29:41.596724 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587919 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 13:29:41.596724 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587922 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 13:29:41.596724 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587925 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 13:29:41.597227 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587927 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 13:29:41.597227 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587930 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 13:29:41.597227 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587933 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 13:29:41.597227 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587937 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 13:29:41.597227 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587941 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 13:29:41.597227 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587944 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 13:29:41.597227 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587947 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 13:29:41.597227 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587950 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 13:29:41.597227 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587953 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 13:29:41.597227 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587956 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 13:29:41.597227 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587958 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 13:29:41.597227 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587961 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 13:29:41.597227 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587964 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 13:29:41.597227 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587966 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 13:29:41.597227 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587969 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 13:29:41.597227 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587973 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 13:29:41.597227 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587976 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 13:29:41.597227 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587979 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 13:29:41.597227 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587982 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 13:29:41.597676 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.587984 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 13:29:41.597676 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.588552 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 13:29:41.597676 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.595280 2573 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 09 13:29:41.597676 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.595297 2573 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 13:29:41.597676 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.596034 2573 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 13:29:41.597676 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.596043 2573 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 13:29:41.597676 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596106 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 13:29:41.597676 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596111 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 13:29:41.597676 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596115 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 13:29:41.597676 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596118 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 13:29:41.597676 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596121 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 13:29:41.597676 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596125 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 13:29:41.597676 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596127 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 13:29:41.597676 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596130 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 13:29:41.598068 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596133 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 13:29:41.598068 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596135 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 13:29:41.598068 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596138 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 13:29:41.598068 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596141 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 13:29:41.598068 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596143 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 13:29:41.598068 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596146 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 13:29:41.598068 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596149 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 13:29:41.598068 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596152 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 13:29:41.598068 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596154 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 13:29:41.598068 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596157 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 13:29:41.598068 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596160 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 13:29:41.598068 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596162 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 13:29:41.598068 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596165 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 13:29:41.598068 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596167 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 13:29:41.598068 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596170 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 13:29:41.598068 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596172 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 13:29:41.598068 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596175 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 13:29:41.598068 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596177 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 13:29:41.598068 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596180 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 13:29:41.598068 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596182 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 13:29:41.598539 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596185 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 13:29:41.598539 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596189 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 13:29:41.598539 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596192 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 13:29:41.598539 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596194 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 13:29:41.598539 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596197 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 13:29:41.598539 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596199 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 13:29:41.598539 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596203 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 13:29:41.598539 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596207 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 13:29:41.598539 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596210 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 13:29:41.598539 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596213 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 13:29:41.598539 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596216 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 13:29:41.598539 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596219 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 13:29:41.598539 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596221 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 13:29:41.598539 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596224 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 09 13:29:41.598539 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596227 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 13:29:41.598539 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596230 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 13:29:41.598539 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596232 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 13:29:41.598539 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596235 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 13:29:41.598539 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596237 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 13:29:41.598539 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596240 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 13:29:41.599065 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596244 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 13:29:41.599065 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596247 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 13:29:41.599065 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596250 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 13:29:41.599065 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596253 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 13:29:41.599065 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596256 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 13:29:41.599065 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596259 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 13:29:41.599065 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596262 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 13:29:41.599065 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596265 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 13:29:41.599065 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596268 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 13:29:41.599065 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596271 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 13:29:41.599065 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596274 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 13:29:41.599065 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596277 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 13:29:41.599065 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596279 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 13:29:41.599065 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596283 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 13:29:41.599065 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596286 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 13:29:41.599065 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596289 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 13:29:41.599065 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596291 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 13:29:41.599065 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596293 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 13:29:41.599065 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596296 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 13:29:41.599526 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596299 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 13:29:41.599526 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596302 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 13:29:41.599526 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596304 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 13:29:41.599526 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596307 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 13:29:41.599526 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596309 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 13:29:41.599526 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596312 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 13:29:41.599526 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596314 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 13:29:41.599526 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596317 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 13:29:41.599526 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596319 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 13:29:41.599526 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596322 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 13:29:41.599526 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596325 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 13:29:41.599526 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596329 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 13:29:41.599526 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596332 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 13:29:41.599526 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596335 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 13:29:41.599526 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596337 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 13:29:41.599526 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596340 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 13:29:41.599526 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596342 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 13:29:41.599526 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596345 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 13:29:41.599526 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596347 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 13:29:41.600073 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596350 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 13:29:41.600073 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596353 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 13:29:41.600073 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596355 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 13:29:41.600073 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596358 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 13:29:41.600073 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596360 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 13:29:41.600073 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596363 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 13:29:41.600073 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596366 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 13:29:41.600073 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596369 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 13:29:41.600073 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596371 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 13:29:41.600073 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596375 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 13:29:41.600073 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596378 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 13:29:41.600073 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596380 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 13:29:41.600073 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596383 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 13:29:41.600073 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.596388 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 13:29:41.600073 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596508 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 13:29:41.600444 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596514 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 13:29:41.600444 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596517 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 13:29:41.600444 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596521 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 13:29:41.600444 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596525 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 13:29:41.600444 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596528 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 09 13:29:41.600444 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596531 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 13:29:41.600444 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596534 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 13:29:41.600444 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596537 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 13:29:41.600444 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596539 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 13:29:41.600444 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596542 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 13:29:41.600444 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596545 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 13:29:41.600444 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596548 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 13:29:41.600444 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596550 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 13:29:41.600444 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596553 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 13:29:41.600444 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596555 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 13:29:41.600444 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596558 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 13:29:41.600444 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596560 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 13:29:41.600444 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596563 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 13:29:41.600444 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596565 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 13:29:41.600904 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596568 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 13:29:41.600904 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596570 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 13:29:41.600904 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596573 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 13:29:41.600904 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596576 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 13:29:41.600904 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596580 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 13:29:41.600904 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596583 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 13:29:41.600904 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596586 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 13:29:41.600904 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596589 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 13:29:41.600904 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596593 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 13:29:41.600904 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596597 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 13:29:41.600904 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596600 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 13:29:41.600904 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596603 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 13:29:41.600904 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596605 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 13:29:41.600904 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596608 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 13:29:41.600904 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596611 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 13:29:41.600904 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596613 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 13:29:41.600904 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596616 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 13:29:41.600904 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596619 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 13:29:41.600904 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596622 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 13:29:41.601361 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596624 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 13:29:41.601361 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596627 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 13:29:41.601361 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596629 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 13:29:41.601361 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596632 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 13:29:41.601361 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596635 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 13:29:41.601361 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596638 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 13:29:41.601361 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596640 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 13:29:41.601361 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596642 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 13:29:41.601361 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596645 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 13:29:41.601361 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596647 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 13:29:41.601361 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596650 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 13:29:41.601361 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596652 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 13:29:41.601361 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596654 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 13:29:41.601361 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596657 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 13:29:41.601361 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596659 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 13:29:41.601361 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596662 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 13:29:41.601361 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596665 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 13:29:41.601361 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596668 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 13:29:41.601361 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596670 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 13:29:41.601361 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596673 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 13:29:41.601830 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596675 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 13:29:41.601830 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596678 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 13:29:41.601830 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596681 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 13:29:41.601830 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596684 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 13:29:41.601830 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596686 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 13:29:41.601830 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596689 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 13:29:41.601830 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596691 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 13:29:41.601830 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596694 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 13:29:41.601830 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596697 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 13:29:41.601830 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596699 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 13:29:41.601830 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596701 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 13:29:41.601830 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596704 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 13:29:41.601830 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596706 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 13:29:41.601830 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596709 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 13:29:41.601830 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596711 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 13:29:41.601830 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596713 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 13:29:41.601830 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596716 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 13:29:41.601830 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596718 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 13:29:41.601830 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596721 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 13:29:41.602309 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596724 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 13:29:41.602309 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596728 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 13:29:41.602309 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596731 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 13:29:41.602309 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596734 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 13:29:41.602309 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596737 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 13:29:41.602309 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596739 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 13:29:41.602309 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596742 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 13:29:41.602309 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596744 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 13:29:41.602309 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596747 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 13:29:41.602309 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596750 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 13:29:41.602309 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596755 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 13:29:41.602309 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596758 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 13:29:41.602309 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596760 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 13:29:41.602309 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596763 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 13:29:41.602309 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596765 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 13:29:41.602309 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596768 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 13:29:41.602309 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596771 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 13:29:41.602309 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596774 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 13:29:41.602309 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596777 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 13:29:41.602816 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596779 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 13:29:41.602816 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:41.596781 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 13:29:41.602816 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.596786 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 13:29:41.602816 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.596931 2573 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 13:29:41.602816 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.598944 2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 13:29:41.602816 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.600138 2573 server.go:1019] "Starting client certificate rotation" Jul 09 13:29:41.602816 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.600244 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 13:29:41.602816 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.600281 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 13:29:41.602816 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.602297 2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 13:29:41.605054 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.605034 2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 13:29:41.616520 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.616494 2573 log.go:25] "Validated CRI v1 runtime API" Jul 09 13:29:41.618830 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.618813 2573 log.go:25] "Validated CRI v1 image API" Jul 09 13:29:41.619629 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.619610 2573 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 13:29:41.622002 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.621980 2573 fs.go:135] Filesystem UUIDs: map[39a5a354-43cf-4c5f-a041-c70e132040ab:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8de70164-c785-40d5-bdb4-d1ae40fc68f6:/dev/nvme0n1p3] Jul 09 13:29:41.622086 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.622000 2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 13:29:41.627034 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.626913 2573 manager.go:216] Machine: {Timestamp:2026-07-09 13:29:41.625648051 +0000 UTC m=+0.291085393 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100749 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21ff8a6c082c6f2293fac66d3442e5 SystemUUID:ec21ff8a-6c08-2c6f-2293-fac66d3442e5 BootID:09b80f06-b11c-4b57-be80-21b4615ba27b Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f6:6b:6c:4c:8f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f6:6b:6c:4c:8f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2a:77:7b:90:6a:69 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 13:29:41.627034 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.627031 2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 13:29:41.627155 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.627117 2573 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 13:29:41.629707 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.629667 2573 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 13:29:41.629854 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.629701 2573 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-87.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 13:29:41.630346 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.630335 2573 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 13:29:41.630381 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.630349 2573 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 13:29:41.630381 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.630356 2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 13:29:41.630920 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.630909 2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 13:29:41.631924 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.631913 2573 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 13:29:41.635367 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.635349 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 13:29:41.635506 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.635495 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 09 13:29:41.635624 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.635615 2573 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 13:29:41.635664 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.635645 2573 kubelet.go:486] "Attempting to sync node with API server" Jul 09 13:29:41.635694 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.635665 2573 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 13:29:41.635694 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.635679 2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 13:29:41.635694 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.635689 2573 kubelet.go:398] "Adding apiserver pod source" Jul 09 13:29:41.635774 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.635702 2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 13:29:41.636098 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.636086 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 13:29:41.636138 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.636133 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 13:29:41.637985 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.637957 2573 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 09 13:29:41.639643 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.639625 2573 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 13:29:41.639692 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.639655 2573 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 13:29:41.640669 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.640652 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 13:29:41.640669 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.640671 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 13:29:41.640787 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.640678 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 13:29:41.640787 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.640684 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 13:29:41.640787 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.640693 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 13:29:41.640787 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.640699 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 13:29:41.640787 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.640711 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 13:29:41.640787 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.640720 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 13:29:41.640787 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.640729 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 13:29:41.640787 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.640734 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 13:29:41.640787 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.640742 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 13:29:41.640787 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.640747 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 13:29:41.641333 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.641322 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 13:29:41.641365 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.641335 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 13:29:41.644665 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.644648 2573 server.go:1268] "Started kubelet" Jul 09 13:29:41.644769 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.644740 2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 13:29:41.645055 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.644911 2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 13:29:41.645232 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.645215 2573 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 13:29:41.645721 ip-10-0-142-87 systemd[1]: Started Kubernetes Kubelet. Jul 09 13:29:41.646457 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.646394 2573 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 13:29:41.646737 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.646618 2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-87.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 13:29:41.646997 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.646962 2573 server.go:310] "Adding debug handlers to kubelet server" Jul 09 13:29:41.647174 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:41.647091 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 13:29:41.647663 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:41.647635 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-87.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 13:29:41.651671 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:41.651648 2573 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 13:29:41.652850 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.652837 2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 13:29:41.652942 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.652923 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 13:29:41.652993 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:41.652979 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-87.ec2.internal\" not found" Jul 09 13:29:41.653441 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.653423 2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 13:29:41.653441 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.653425 2573 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 13:29:41.653628 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.653465 2573 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 13:29:41.653628 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.653556 2573 reconstruct.go:97] "Volume reconstruction finished" Jul 09 13:29:41.653628 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.653562 2573 reconciler.go:29] "Reconciler: start to sync state" Jul 09 13:29:41.653755 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.653644 2573 factory.go:55] Registering systemd factory Jul 09 13:29:41.653755 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.653660 2573 factory.go:223] Registration of the systemd container factory successfully Jul 09 13:29:41.653755 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:41.652821 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-87.ec2.internal.18c0a23080f89285 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-87.ec2.internal,UID:ip-10-0-142-87.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-87.ec2.internal,},FirstTimestamp:2026-07-09 13:29:41.644620421 +0000 UTC m=+0.310057765,LastTimestamp:2026-07-09 13:29:41.644620421 +0000 UTC m=+0.310057765,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-87.ec2.internal,}" Jul 09 13:29:41.653927 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.653918 2573 factory.go:153] Registering CRI-O factory Jul 09 13:29:41.653983 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.653932 2573 factory.go:223] Registration of the crio container factory successfully Jul 09 13:29:41.654047 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.653989 2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 13:29:41.654047 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.654010 2573 factory.go:103] Registering Raw factory Jul 09 13:29:41.654047 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.654023 2573 manager.go:1196] Started watching for new ooms in manager Jul 09 13:29:41.654820 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.654807 2573 manager.go:349] Starting recovery of all containers Jul 09 13:29:41.657156 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:41.657107 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 13:29:41.657249 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:41.657223 2573 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-87.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 13:29:41.667056 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.667022 2573 manager.go:354] Recovery completed Jul 09 13:29:41.668686 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:41.668652 2573 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 09 13:29:41.672657 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.672644 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 13:29:41.674856 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.674835 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-87.ec2.internal" event="NodeHasSufficientMemory" Jul 09 13:29:41.674980 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.674868 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-87.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 13:29:41.674980 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.674899 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-87.ec2.internal" event="NodeHasSufficientPID" Jul 09 13:29:41.675438 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.675423 2573 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 13:29:41.675438 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.675437 2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 13:29:41.675547 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.675444 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 13:29:41.675547 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.675455 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 09 13:29:41.676168 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.676148 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-87.ec2.internal" event="NodeHasSufficientMemory" Jul 09 13:29:41.676233 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.676179 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-87.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 13:29:41.676233 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.676189 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-87.ec2.internal" event="NodeHasSufficientPID" Jul 09 13:29:41.677620 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.677609 2573 policy_none.go:49] "None policy: Start" Jul 09 13:29:41.677662 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.677625 2573 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 13:29:41.677662 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.677636 2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 13:29:41.678195 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:41.678122 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-87.ec2.internal.18c0a23082c5e277 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-87.ec2.internal,UID:ip-10-0-142-87.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-87.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-87.ec2.internal,},FirstTimestamp:2026-07-09 13:29:41.674852983 +0000 UTC m=+0.340290328,LastTimestamp:2026-07-09 13:29:41.674852983 +0000 UTC m=+0.340290328,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-87.ec2.internal,}" Jul 09 13:29:41.679475 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:41.679461 2573 policy_none.go:47] "Start" Jul 09 13:29:41.687409 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:41.687222 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-87.ec2.internal.18c0a23082c6412e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-87.ec2.internal,UID:ip-10-0-142-87.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-87.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-87.ec2.internal,},FirstTimestamp:2026-07-09 13:29:41.67487723 +0000 UTC m=+0.340314572,LastTimestamp:2026-07-09 13:29:41.67487723 +0000 UTC m=+0.340314572,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-87.ec2.internal,}" Jul 09 13:29:41.698395 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:41.698253 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-87.ec2.internal.18c0a23082c6b0f6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-87.ec2.internal,UID:ip-10-0-142-87.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-142-87.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-142-87.ec2.internal,},FirstTimestamp:2026-07-09 13:29:41.674905846 +0000 UTC m=+0.340343192,LastTimestamp:2026-07-09 13:29:41.674905846 +0000 UTC m=+0.340343192,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-87.ec2.internal,}" Jul 09 13:29:42.870174 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:42.870091 2573 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-87.ec2.internal.18c0a23082c5e277\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-87.ec2.internal.18c0a23082c5e277 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-87.ec2.internal,UID:ip-10-0-142-87.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-87.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-87.ec2.internal,},FirstTimestamp:2026-07-09 13:29:41.674852983 +0000 UTC m=+0.340290328,LastTimestamp:2026-07-09 13:29:41.676164669 +0000 UTC m=+0.341602012,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-87.ec2.internal,}" Jul 09 13:29:42.874689 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:42.874670 2573 manager.go:337] "Starting Device Plugin manager" Jul 09 13:29:42.874790 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:42.874715 2573 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 13:29:42.874790 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:42.874728 2573 server.go:87] "Starting device plugin registration server" Jul 09 13:29:42.875033 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:42.875005 2573 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 13:29:42.875092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:42.875035 2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 13:29:42.875149 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:42.875130 2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 13:29:42.875235 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:42.875217 2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 13:29:42.875235 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:42.875235 2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 13:29:42.875782 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:42.875765 2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 13:29:42.875863 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:42.875811 2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-87.ec2.internal\" not found" Jul 09 13:29:42.877122 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:42.877104 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ft7zv" Jul 09 13:29:42.879155 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:42.879036 2573 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-87.ec2.internal.18c0a23082c6412e\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-87.ec2.internal.18c0a23082c6412e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-87.ec2.internal,UID:ip-10-0-142-87.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-87.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-87.ec2.internal,},FirstTimestamp:2026-07-09 13:29:41.67487723 +0000 UTC m=+0.340314572,LastTimestamp:2026-07-09 13:29:41.676184141 +0000 UTC m=+0.341621482,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-87.ec2.internal,}" Jul 09 13:29:42.886790 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:42.886772 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ft7zv" Jul 09 13:29:42.888679 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:42.888601 2573 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-87.ec2.internal.18c0a23082c6b0f6\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-87.ec2.internal.18c0a23082c6b0f6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-87.ec2.internal,UID:ip-10-0-142-87.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-142-87.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-142-87.ec2.internal,},FirstTimestamp:2026-07-09 13:29:41.674905846 +0000 UTC m=+0.340343192,LastTimestamp:2026-07-09 13:29:41.676193148 +0000 UTC m=+0.341630490,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-87.ec2.internal,}" Jul 09 13:29:42.937915 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:42.937877 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 13:29:42.939920 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:42.939141 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 13:29:42.939920 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:42.939176 2573 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 13:29:42.939920 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:42.939198 2573 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 13:29:42.939920 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:42.939231 2573 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 13:29:42.941993 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:42.941975 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 13:29:42.976172 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:42.976139 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 13:29:42.977255 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:42.977233 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-87.ec2.internal" event="NodeHasSufficientMemory" Jul 09 13:29:42.977369 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:42.977270 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-87.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 13:29:42.977369 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:42.977281 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-87.ec2.internal" event="NodeHasSufficientPID" Jul 09 13:29:42.977369 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:42.977305 2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-87.ec2.internal" Jul 09 13:29:42.987163 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:42.987111 2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-87.ec2.internal" Jul 09 13:29:42.987163 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:42.987136 2573 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-87.ec2.internal\": node \"ip-10-0-142-87.ec2.internal\" not found" Jul 09 13:29:42.999658 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:42.999633 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-87.ec2.internal\" not found" Jul 09 13:29:43.040285 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.040251 2573 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-87.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-87.ec2.internal"] Jul 09 13:29:43.040453 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.040352 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 13:29:43.042598 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.042579 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-87.ec2.internal" event="NodeHasSufficientMemory" Jul 09 13:29:43.042722 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.042612 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-87.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 13:29:43.042722 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.042622 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-87.ec2.internal" event="NodeHasSufficientPID" Jul 09 13:29:43.044716 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.044702 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 13:29:43.044879 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.044863 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-87.ec2.internal" Jul 09 13:29:43.044943 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.044904 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 13:29:43.045502 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.045487 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-87.ec2.internal" event="NodeHasSufficientMemory" Jul 09 13:29:43.045502 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.045497 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-87.ec2.internal" event="NodeHasSufficientMemory" Jul 09 13:29:43.045630 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.045513 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-87.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 13:29:43.045630 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.045527 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-87.ec2.internal" event="NodeHasSufficientPID" Jul 09 13:29:43.045630 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.045514 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-87.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 13:29:43.045630 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.045606 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-87.ec2.internal" event="NodeHasSufficientPID" Jul 09 13:29:43.047646 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.047626 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-87.ec2.internal" Jul 09 13:29:43.047748 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.047661 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 13:29:43.048718 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.048689 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-87.ec2.internal" event="NodeHasSufficientMemory" Jul 09 13:29:43.048718 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.048718 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-87.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 13:29:43.048880 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.048733 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-87.ec2.internal" event="NodeHasSufficientPID" Jul 09 13:29:43.063662 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:43.063636 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-87.ec2.internal\" not found" node="ip-10-0-142-87.ec2.internal" Jul 09 13:29:43.067705 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:43.067686 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-87.ec2.internal\" not found" node="ip-10-0-142-87.ec2.internal" Jul 09 13:29:43.099998 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:43.099969 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-87.ec2.internal\" not found" Jul 09 13:29:43.115313 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.115287 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a3649323ce8714df59b0d7e58d8cfecb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-87.ec2.internal\" (UID: \"a3649323ce8714df59b0d7e58d8cfecb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-87.ec2.internal" Jul 09 13:29:43.115399 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.115320 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3649323ce8714df59b0d7e58d8cfecb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-87.ec2.internal\" (UID: \"a3649323ce8714df59b0d7e58d8cfecb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-87.ec2.internal" Jul 09 13:29:43.115399 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.115338 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dc8175f5b3169ac412d470dea8cf3d3d-config\") pod \"kube-apiserver-proxy-ip-10-0-142-87.ec2.internal\" (UID: \"dc8175f5b3169ac412d470dea8cf3d3d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-87.ec2.internal" Jul 09 13:29:43.200206 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:43.200179 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-87.ec2.internal\" not found" Jul 09 13:29:43.215597 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.215562 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a3649323ce8714df59b0d7e58d8cfecb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-87.ec2.internal\" (UID: \"a3649323ce8714df59b0d7e58d8cfecb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-87.ec2.internal" Jul 09 13:29:43.215666 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.215604 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3649323ce8714df59b0d7e58d8cfecb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-87.ec2.internal\" (UID: \"a3649323ce8714df59b0d7e58d8cfecb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-87.ec2.internal" Jul 09 13:29:43.215666 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.215628 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dc8175f5b3169ac412d470dea8cf3d3d-config\") pod \"kube-apiserver-proxy-ip-10-0-142-87.ec2.internal\" (UID: \"dc8175f5b3169ac412d470dea8cf3d3d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-87.ec2.internal" Jul 09 13:29:43.215751 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.215678 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dc8175f5b3169ac412d470dea8cf3d3d-config\") pod \"kube-apiserver-proxy-ip-10-0-142-87.ec2.internal\" (UID: \"dc8175f5b3169ac412d470dea8cf3d3d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-87.ec2.internal" Jul 09 13:29:43.215751 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.215678 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a3649323ce8714df59b0d7e58d8cfecb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-87.ec2.internal\" (UID: \"a3649323ce8714df59b0d7e58d8cfecb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-87.ec2.internal" Jul 09 13:29:43.215751 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.215707 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3649323ce8714df59b0d7e58d8cfecb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-87.ec2.internal\" (UID: \"a3649323ce8714df59b0d7e58d8cfecb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-87.ec2.internal" Jul 09 13:29:43.301071 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:43.300980 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-87.ec2.internal\" not found" Jul 09 13:29:43.366451 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.366419 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-87.ec2.internal" Jul 09 13:29:43.369777 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.369752 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-87.ec2.internal" Jul 09 13:29:43.401996 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:43.401955 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-87.ec2.internal\" not found" Jul 09 13:29:43.502462 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:43.502413 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-87.ec2.internal\" not found" Jul 09 13:29:43.603056 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:43.602942 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-87.ec2.internal\" not found" Jul 09 13:29:43.703507 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:43.703457 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-87.ec2.internal\" not found" Jul 09 13:29:43.737111 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.737090 2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 13:29:43.759512 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.759485 2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 13:29:43.759649 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.759633 2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 13:29:43.759684 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.759664 2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 13:29:43.803706 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:43.803664 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-87.ec2.internal\" not found" Jul 09 13:29:43.803914 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.803900 2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 13:29:43.814191 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.814165 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-87.ec2.internal" Jul 09 13:29:43.814191 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.814181 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 13:29:43.826600 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.826575 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 13:29:43.828235 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.828218 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-87.ec2.internal" Jul 09 13:29:43.831191 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.831173 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 13:29:43.844336 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.844311 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 13:29:43.858563 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.858504 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8w9cx" Jul 09 13:29:43.868449 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.868426 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8w9cx" Jul 09 13:29:43.891069 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.891030 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 13:24:42 +0000 UTC" deadline="2028-03-30 21:03:43.850584802 +0000 UTC" Jul 09 13:29:43.891069 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.891062 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15127h33m59.959525933s" Jul 09 13:29:43.918825 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:43.918793 2573 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 13:29:44.102354 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:44.102311 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3649323ce8714df59b0d7e58d8cfecb.slice/crio-69f43870a685d3e5ed08c82a49b52bc1dfb98a1f63424777b8bccdeeb4f867b3 WatchSource:0}: Error finding container 69f43870a685d3e5ed08c82a49b52bc1dfb98a1f63424777b8bccdeeb4f867b3: Status 404 returned error can't find the container with id 69f43870a685d3e5ed08c82a49b52bc1dfb98a1f63424777b8bccdeeb4f867b3 Jul 09 13:29:44.102705 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:44.102682 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc8175f5b3169ac412d470dea8cf3d3d.slice/crio-b07579aebedc3b034084cedefbb625864740ad8c82e4b0a90267fbb3570af513 WatchSource:0}: Error finding container b07579aebedc3b034084cedefbb625864740ad8c82e4b0a90267fbb3570af513: Status 404 returned error can't find the container with id b07579aebedc3b034084cedefbb625864740ad8c82e4b0a90267fbb3570af513 Jul 09 13:29:44.106516 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.106499 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 13:29:44.741853 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.741821 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 13:29:44.797202 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.797165 2573 apiserver.go:52] "Watching apiserver" Jul 09 13:29:44.809389 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.809356 2573 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 13:29:44.809844 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.809811 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-k5fs2","openshift-image-registry/node-ca-fcwpf","openshift-multus/multus-qb95w","openshift-network-diagnostics/network-check-target-h6qqn","openshift-network-operator/iptables-alerter-xnbzz","openshift-ovn-kubernetes/ovnkube-node-vmms8","kube-system/global-pull-secret-syncer-65fnv","kube-system/konnectivity-agent-tnf9v","openshift-cluster-node-tuning-operator/tuned-cmb9f","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-87.ec2.internal","openshift-multus/multus-additional-cni-plugins-v7rlv","openshift-multus/network-metrics-daemon-5zl2x","kube-system/kube-apiserver-proxy-ip-10-0-142-87.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd"] Jul 09 13:29:44.812459 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.812428 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xnbzz" Jul 09 13:29:44.814834 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.814812 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tnf9v" Jul 09 13:29:44.815577 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.815299 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 13:29:44.815577 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.815540 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 13:29:44.816364 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.816343 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 13:29:44.817149 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.817101 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:29:44.817247 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:44.817178 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65fnv" podUID="fb20e4df-4233-4fb6-a1f7-e58a789e3d56" Jul 09 13:29:44.817616 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.817595 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nxw6r\"" Jul 09 13:29:44.818009 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.817980 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-v57kt\"" Jul 09 13:29:44.818221 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.818202 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 13:29:44.822211 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.822039 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:29:44.822211 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:44.822138 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6qqn" podUID="9e8320e4-2106-4edf-b225-d65a4b3eae55" Jul 09 13:29:44.822977 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.822727 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 13:29:44.824388 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.824360 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b3cd6f5c-b27b-4312-84d2-1e706cb7af29-host-slash\") pod \"iptables-alerter-xnbzz\" (UID: \"b3cd6f5c-b27b-4312-84d2-1e706cb7af29\") " pod="openshift-network-operator/iptables-alerter-xnbzz" Jul 09 13:29:44.824492 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.824393 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56qlm\" (UniqueName: \"kubernetes.io/projected/b3cd6f5c-b27b-4312-84d2-1e706cb7af29-kube-api-access-56qlm\") pod \"iptables-alerter-xnbzz\" (UID: \"b3cd6f5c-b27b-4312-84d2-1e706cb7af29\") " pod="openshift-network-operator/iptables-alerter-xnbzz" Jul 09 13:29:44.824492 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.824420 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ef1861a0-ca40-4753-80c2-7085d1884d07-agent-certs\") pod \"konnectivity-agent-tnf9v\" (UID: \"ef1861a0-ca40-4753-80c2-7085d1884d07\") " pod="kube-system/konnectivity-agent-tnf9v" Jul 09 13:29:44.824492 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.824441 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ef1861a0-ca40-4753-80c2-7085d1884d07-konnectivity-ca\") pod \"konnectivity-agent-tnf9v\" (UID: \"ef1861a0-ca40-4753-80c2-7085d1884d07\") " pod="kube-system/konnectivity-agent-tnf9v" Jul 09 13:29:44.824492 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.824469 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b3cd6f5c-b27b-4312-84d2-1e706cb7af29-iptables-alerter-script\") pod \"iptables-alerter-xnbzz\" (UID: \"b3cd6f5c-b27b-4312-84d2-1e706cb7af29\") " pod="openshift-network-operator/iptables-alerter-xnbzz" Jul 09 13:29:44.824687 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.824610 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qb95w" Jul 09 13:29:44.827531 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.827474 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:44.830364 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.829368 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xlfqp\"" Jul 09 13:29:44.830364 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.829604 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 13:29:44.830364 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.829792 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 13:29:44.830364 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.830003 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 13:29:44.830364 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.830209 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 13:29:44.830779 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.830412 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k5fs2" Jul 09 13:29:44.830779 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.830485 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 13:29:44.830779 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.830502 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 13:29:44.830911 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.830787 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 13:29:44.831374 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.831201 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 13:29:44.831374 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.831209 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-d8h2x\"" Jul 09 13:29:44.831519 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.831438 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 13:29:44.832267 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.832246 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fcwpf" Jul 09 13:29:44.833042 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.832993 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 13:29:44.835069 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.834656 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:44.836247 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.836225 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 13:29:44.836779 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.836759 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 13:29:44.837583 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.837003 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5svrk\"" Jul 09 13:29:44.837583 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.837296 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 13:29:44.837583 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.837495 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 13:29:44.837583 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.837554 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qjv59\"" Jul 09 13:29:44.837839 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.837680 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 13:29:44.837839 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.837822 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-92tr2\"" Jul 09 13:29:44.838133 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.838114 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:44.839436 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.839389 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 13:29:44.839436 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.839395 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 13:29:44.840134 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.839894 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:29:44.840134 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:44.839959 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5zl2x" podUID="fbe44bc5-d23a-4535-b1d8-74ec52d0ce47" Jul 09 13:29:44.843200 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.842800 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" Jul 09 13:29:44.845754 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.845180 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 13:29:44.845754 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.845489 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-m77sp\"" Jul 09 13:29:44.849340 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.845981 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 13:29:44.849340 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.847317 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 13:29:44.849340 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.847346 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 13:29:44.849340 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.847655 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 13:29:44.849340 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.848394 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nz8br\"" Jul 09 13:29:44.873528 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.872231 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 13:24:43 +0000 UTC" deadline="2028-01-02 19:14:58.809959131 +0000 UTC" Jul 09 13:29:44.873528 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.872282 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13013h45m13.937681287s" Jul 09 13:29:44.915493 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.915466 2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 13:29:44.925344 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.925312 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a74fe7a2-2170-47f0-aad8-6764d8ab41da-socket-dir\") pod \"aws-ebs-csi-driver-node-xjchd\" (UID: \"a74fe7a2-2170-47f0-aad8-6764d8ab41da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" Jul 09 13:29:44.925515 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.925356 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtpfd\" (UniqueName: \"kubernetes.io/projected/df9fed4f-2388-4d2e-abca-f91fc71334bc-kube-api-access-dtpfd\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:44.925515 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.925388 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3b512d09-0288-4fea-864e-fdc734afb1bd-ovn-node-metrics-cert\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:44.925515 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.925415 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-etc-tuned\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:44.925515 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.925439 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a74fe7a2-2170-47f0-aad8-6764d8ab41da-etc-selinux\") pod \"aws-ebs-csi-driver-node-xjchd\" (UID: \"a74fe7a2-2170-47f0-aad8-6764d8ab41da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" Jul 09 13:29:44.925515 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.925466 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjqpc\" (UniqueName: \"kubernetes.io/projected/9e8320e4-2106-4edf-b225-d65a4b3eae55-kube-api-access-wjqpc\") pod \"network-check-target-h6qqn\" (UID: \"9e8320e4-2106-4edf-b225-d65a4b3eae55\") " pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:29:44.925515 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.925489 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-etc-kubernetes\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:44.925515 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.925513 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-etc-openvswitch\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:44.925853 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.925533 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-sys\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:44.925853 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.925548 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8556c373-2549-4328-aa41-2a05076a088a-cnibin\") pod \"multus-additional-cni-plugins-v7rlv\" (UID: \"8556c373-2549-4328-aa41-2a05076a088a\") " pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:44.925853 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.925570 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56qlm\" (UniqueName: \"kubernetes.io/projected/b3cd6f5c-b27b-4312-84d2-1e706cb7af29-kube-api-access-56qlm\") pod \"iptables-alerter-xnbzz\" (UID: \"b3cd6f5c-b27b-4312-84d2-1e706cb7af29\") " pod="openshift-network-operator/iptables-alerter-xnbzz" Jul 09 13:29:44.925853 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.925592 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a74fe7a2-2170-47f0-aad8-6764d8ab41da-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xjchd\" (UID: \"a74fe7a2-2170-47f0-aad8-6764d8ab41da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" Jul 09 13:29:44.925853 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.925612 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-hostroot\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:44.925853 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.925635 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-etc-sysconfig\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:44.925853 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.925658 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-dbus\") pod \"global-pull-secret-syncer-65fnv\" (UID: \"fb20e4df-4233-4fb6-a1f7-e58a789e3d56\") " pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:29:44.925853 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.925680 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7fc8a7a4-1f79-475a-9eed-31141361e131-serviceca\") pod \"node-ca-fcwpf\" (UID: \"7fc8a7a4-1f79-475a-9eed-31141361e131\") " pod="openshift-image-registry/node-ca-fcwpf" Jul 09 13:29:44.925853 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.925706 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ef1861a0-ca40-4753-80c2-7085d1884d07-konnectivity-ca\") pod \"konnectivity-agent-tnf9v\" (UID: \"ef1861a0-ca40-4753-80c2-7085d1884d07\") " pod="kube-system/konnectivity-agent-tnf9v" Jul 09 13:29:44.925853 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.925737 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzn62\" (UniqueName: \"kubernetes.io/projected/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-kube-api-access-wzn62\") pod \"network-metrics-daemon-5zl2x\" (UID: \"fbe44bc5-d23a-4535-b1d8-74ec52d0ce47\") " pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:29:44.925853 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.925760 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-os-release\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:44.926384 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.925847 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-host-run-k8s-cni-cncf-io\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:44.926384 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.925889 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-run-systemd\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:44.926384 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.925922 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-host-cni-netd\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:44.926384 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.925949 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8556c373-2549-4328-aa41-2a05076a088a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v7rlv\" (UID: \"8556c373-2549-4328-aa41-2a05076a088a\") " pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:44.926384 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.925974 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-kubelet-config\") pod \"global-pull-secret-syncer-65fnv\" (UID: \"fb20e4df-4233-4fb6-a1f7-e58a789e3d56\") " pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:29:44.926384 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.925998 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a74fe7a2-2170-47f0-aad8-6764d8ab41da-sys-fs\") pod \"aws-ebs-csi-driver-node-xjchd\" (UID: \"a74fe7a2-2170-47f0-aad8-6764d8ab41da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" Jul 09 13:29:44.926384 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926038 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k247d\" (UniqueName: \"kubernetes.io/projected/a74fe7a2-2170-47f0-aad8-6764d8ab41da-kube-api-access-k247d\") pod \"aws-ebs-csi-driver-node-xjchd\" (UID: \"a74fe7a2-2170-47f0-aad8-6764d8ab41da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" Jul 09 13:29:44.926384 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926062 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-cnibin\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:44.926384 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926125 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-host-run-netns\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:44.926384 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926149 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-host-var-lib-kubelet\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:44.926384 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926180 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c9f5a139-bec5-4097-8ca2-0618cd70bc56-hosts-file\") pod \"node-resolver-k5fs2\" (UID: \"c9f5a139-bec5-4097-8ca2-0618cd70bc56\") " pod="openshift-dns/node-resolver-k5fs2" Jul 09 13:29:44.926384 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926202 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9f5a139-bec5-4097-8ca2-0618cd70bc56-tmp-dir\") pod \"node-resolver-k5fs2\" (UID: \"c9f5a139-bec5-4097-8ca2-0618cd70bc56\") " pod="openshift-dns/node-resolver-k5fs2" Jul 09 13:29:44.926384 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926224 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-etc-kubernetes\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:44.926384 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926249 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-metrics-certs\") pod \"network-metrics-daemon-5zl2x\" (UID: \"fbe44bc5-d23a-4535-b1d8-74ec52d0ce47\") " pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:29:44.926384 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926272 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-host-var-lib-cni-bin\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:44.926384 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926295 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-host-kubelet\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:44.927146 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926325 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-log-socket\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:44.927146 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926356 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3b512d09-0288-4fea-864e-fdc734afb1bd-ovnkube-config\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:44.927146 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926382 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8556c373-2549-4328-aa41-2a05076a088a-os-release\") pod \"multus-additional-cni-plugins-v7rlv\" (UID: \"8556c373-2549-4328-aa41-2a05076a088a\") " pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:44.927146 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926403 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7fc8a7a4-1f79-475a-9eed-31141361e131-host\") pod \"node-ca-fcwpf\" (UID: \"7fc8a7a4-1f79-475a-9eed-31141361e131\") " pod="openshift-image-registry/node-ca-fcwpf" Jul 09 13:29:44.927146 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926425 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-host-slash\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:44.927146 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926447 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ef1861a0-ca40-4753-80c2-7085d1884d07-konnectivity-ca\") pod \"konnectivity-agent-tnf9v\" (UID: \"ef1861a0-ca40-4753-80c2-7085d1884d07\") " pod="kube-system/konnectivity-agent-tnf9v" Jul 09 13:29:44.927146 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926449 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:44.927146 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926518 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-var-lib-kubelet\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:44.927146 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926547 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpvdp\" (UniqueName: \"kubernetes.io/projected/7fc8a7a4-1f79-475a-9eed-31141361e131-kube-api-access-vpvdp\") pod \"node-ca-fcwpf\" (UID: \"7fc8a7a4-1f79-475a-9eed-31141361e131\") " pod="openshift-image-registry/node-ca-fcwpf" Jul 09 13:29:44.927146 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926569 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-multus-cni-dir\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:44.927146 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926598 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-run-openvswitch\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:44.927146 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926620 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gddm\" (UniqueName: \"kubernetes.io/projected/c9f5a139-bec5-4097-8ca2-0618cd70bc56-kube-api-access-9gddm\") pod \"node-resolver-k5fs2\" (UID: \"c9f5a139-bec5-4097-8ca2-0618cd70bc56\") " pod="openshift-dns/node-resolver-k5fs2" Jul 09 13:29:44.927146 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926641 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-lib-modules\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:44.927146 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926669 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8556c373-2549-4328-aa41-2a05076a088a-cni-binary-copy\") pod \"multus-additional-cni-plugins-v7rlv\" (UID: \"8556c373-2549-4328-aa41-2a05076a088a\") " pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:44.927146 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926684 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/df9fed4f-2388-4d2e-abca-f91fc71334bc-cni-binary-copy\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:44.927146 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926698 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-systemd-units\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:44.927928 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926742 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-var-lib-openvswitch\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:44.927928 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926777 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-run-ovn\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:44.927928 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926808 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3b512d09-0288-4fea-864e-fdc734afb1bd-env-overrides\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:44.927928 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926857 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmxxn\" (UniqueName: \"kubernetes.io/projected/3b512d09-0288-4fea-864e-fdc734afb1bd-kube-api-access-tmxxn\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:44.927928 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926894 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-etc-modprobe-d\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:44.927928 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926929 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-run\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:44.927928 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926954 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/df9fed4f-2388-4d2e-abca-f91fc71334bc-multus-daemon-config\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:44.927928 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.926983 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-etc-systemd\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:44.927928 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927003 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-tmp\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:44.927928 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927056 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8556c373-2549-4328-aa41-2a05076a088a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v7rlv\" (UID: \"8556c373-2549-4328-aa41-2a05076a088a\") " pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:44.927928 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927089 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-original-pull-secret\") pod \"global-pull-secret-syncer-65fnv\" (UID: \"fb20e4df-4233-4fb6-a1f7-e58a789e3d56\") " pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:29:44.927928 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927128 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b3cd6f5c-b27b-4312-84d2-1e706cb7af29-host-slash\") pod \"iptables-alerter-xnbzz\" (UID: \"b3cd6f5c-b27b-4312-84d2-1e706cb7af29\") " pod="openshift-network-operator/iptables-alerter-xnbzz" Jul 09 13:29:44.927928 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927166 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-multus-socket-dir-parent\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:44.927928 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927191 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-host-var-lib-cni-multus\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:44.927928 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927195 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b3cd6f5c-b27b-4312-84d2-1e706cb7af29-host-slash\") pod \"iptables-alerter-xnbzz\" (UID: \"b3cd6f5c-b27b-4312-84d2-1e706cb7af29\") " pod="openshift-network-operator/iptables-alerter-xnbzz" Jul 09 13:29:44.927928 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927222 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b3cd6f5c-b27b-4312-84d2-1e706cb7af29-iptables-alerter-script\") pod \"iptables-alerter-xnbzz\" (UID: \"b3cd6f5c-b27b-4312-84d2-1e706cb7af29\") " pod="openshift-network-operator/iptables-alerter-xnbzz" Jul 09 13:29:44.928694 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927251 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3b512d09-0288-4fea-864e-fdc734afb1bd-ovnkube-script-lib\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:44.928694 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927315 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8556c373-2549-4328-aa41-2a05076a088a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v7rlv\" (UID: \"8556c373-2549-4328-aa41-2a05076a088a\") " pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:44.928694 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927340 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlzsz\" (UniqueName: \"kubernetes.io/projected/8556c373-2549-4328-aa41-2a05076a088a-kube-api-access-dlzsz\") pod \"multus-additional-cni-plugins-v7rlv\" (UID: \"8556c373-2549-4328-aa41-2a05076a088a\") " pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:44.928694 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927368 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a74fe7a2-2170-47f0-aad8-6764d8ab41da-device-dir\") pod \"aws-ebs-csi-driver-node-xjchd\" (UID: \"a74fe7a2-2170-47f0-aad8-6764d8ab41da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" Jul 09 13:29:44.928694 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927389 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-system-cni-dir\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:44.928694 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927410 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-host-cni-bin\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:44.928694 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927438 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ef1861a0-ca40-4753-80c2-7085d1884d07-agent-certs\") pod \"konnectivity-agent-tnf9v\" (UID: \"ef1861a0-ca40-4753-80c2-7085d1884d07\") " pod="kube-system/konnectivity-agent-tnf9v" Jul 09 13:29:44.928694 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927499 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-multus-conf-dir\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:44.928694 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927523 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-host-run-netns\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:44.928694 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927547 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-host-run-ovn-kubernetes\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:44.928694 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927571 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-host\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:44.928694 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927601 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-node-log\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:44.928694 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927633 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-etc-sysctl-conf\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:44.928694 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927684 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpbvn\" (UniqueName: \"kubernetes.io/projected/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-kube-api-access-wpbvn\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:44.928694 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927723 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8556c373-2549-4328-aa41-2a05076a088a-system-cni-dir\") pod \"multus-additional-cni-plugins-v7rlv\" (UID: \"8556c373-2549-4328-aa41-2a05076a088a\") " pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:44.928694 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927811 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b3cd6f5c-b27b-4312-84d2-1e706cb7af29-iptables-alerter-script\") pod \"iptables-alerter-xnbzz\" (UID: \"b3cd6f5c-b27b-4312-84d2-1e706cb7af29\") " pod="openshift-network-operator/iptables-alerter-xnbzz" Jul 09 13:29:44.929482 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927833 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a74fe7a2-2170-47f0-aad8-6764d8ab41da-registration-dir\") pod \"aws-ebs-csi-driver-node-xjchd\" (UID: \"a74fe7a2-2170-47f0-aad8-6764d8ab41da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" Jul 09 13:29:44.929482 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927867 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-host-run-multus-certs\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:44.929482 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.927913 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-etc-sysctl-d\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:44.929482 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.928169 2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 13:29:44.936381 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.936349 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ef1861a0-ca40-4753-80c2-7085d1884d07-agent-certs\") pod \"konnectivity-agent-tnf9v\" (UID: \"ef1861a0-ca40-4753-80c2-7085d1884d07\") " pod="kube-system/konnectivity-agent-tnf9v" Jul 09 13:29:44.945553 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.945523 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56qlm\" (UniqueName: \"kubernetes.io/projected/b3cd6f5c-b27b-4312-84d2-1e706cb7af29-kube-api-access-56qlm\") pod \"iptables-alerter-xnbzz\" (UID: \"b3cd6f5c-b27b-4312-84d2-1e706cb7af29\") " pod="openshift-network-operator/iptables-alerter-xnbzz" Jul 09 13:29:44.946538 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.946487 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-87.ec2.internal" event={"ID":"a3649323ce8714df59b0d7e58d8cfecb","Type":"ContainerStarted","Data":"69f43870a685d3e5ed08c82a49b52bc1dfb98a1f63424777b8bccdeeb4f867b3"} Jul 09 13:29:44.948997 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:44.948967 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-87.ec2.internal" event={"ID":"dc8175f5b3169ac412d470dea8cf3d3d","Type":"ContainerStarted","Data":"b07579aebedc3b034084cedefbb625864740ad8c82e4b0a90267fbb3570af513"} Jul 09 13:29:45.028541 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.028445 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-run-openvswitch\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.028541 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.028514 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-run-openvswitch\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.028781 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.028543 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gddm\" (UniqueName: \"kubernetes.io/projected/c9f5a139-bec5-4097-8ca2-0618cd70bc56-kube-api-access-9gddm\") pod \"node-resolver-k5fs2\" (UID: \"c9f5a139-bec5-4097-8ca2-0618cd70bc56\") " pod="openshift-dns/node-resolver-k5fs2" Jul 09 13:29:45.028781 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.028585 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-lib-modules\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.028781 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.028610 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8556c373-2549-4328-aa41-2a05076a088a-cni-binary-copy\") pod \"multus-additional-cni-plugins-v7rlv\" (UID: \"8556c373-2549-4328-aa41-2a05076a088a\") " pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:45.028781 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.028640 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/df9fed4f-2388-4d2e-abca-f91fc71334bc-cni-binary-copy\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.028781 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.028674 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-systemd-units\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.028781 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.028714 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-var-lib-openvswitch\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.028781 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.028737 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-run-ovn\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.028781 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.028760 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3b512d09-0288-4fea-864e-fdc734afb1bd-env-overrides\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.028781 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.028758 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-lib-modules\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.029228 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.028790 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmxxn\" (UniqueName: \"kubernetes.io/projected/3b512d09-0288-4fea-864e-fdc734afb1bd-kube-api-access-tmxxn\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.029228 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.028817 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-etc-modprobe-d\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.029228 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.028819 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-var-lib-openvswitch\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.029228 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.028845 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-run\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.029228 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.028874 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/df9fed4f-2388-4d2e-abca-f91fc71334bc-multus-daemon-config\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.029228 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.028881 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-systemd-units\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.029228 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.028902 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-etc-systemd\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.029228 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.028924 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-tmp\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.029228 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.028948 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8556c373-2549-4328-aa41-2a05076a088a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v7rlv\" (UID: \"8556c373-2549-4328-aa41-2a05076a088a\") " pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:45.029228 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.028972 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-original-pull-secret\") pod \"global-pull-secret-syncer-65fnv\" (UID: \"fb20e4df-4233-4fb6-a1f7-e58a789e3d56\") " pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:29:45.029228 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029000 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-multus-socket-dir-parent\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.029228 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029035 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-host-var-lib-cni-multus\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.029228 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029058 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3b512d09-0288-4fea-864e-fdc734afb1bd-ovnkube-script-lib\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.029228 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029075 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8556c373-2549-4328-aa41-2a05076a088a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v7rlv\" (UID: \"8556c373-2549-4328-aa41-2a05076a088a\") " pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:45.029228 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029089 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlzsz\" (UniqueName: \"kubernetes.io/projected/8556c373-2549-4328-aa41-2a05076a088a-kube-api-access-dlzsz\") pod \"multus-additional-cni-plugins-v7rlv\" (UID: \"8556c373-2549-4328-aa41-2a05076a088a\") " pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:45.029228 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029145 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-run-ovn\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.029228 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029171 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a74fe7a2-2170-47f0-aad8-6764d8ab41da-device-dir\") pod \"aws-ebs-csi-driver-node-xjchd\" (UID: \"a74fe7a2-2170-47f0-aad8-6764d8ab41da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" Jul 09 13:29:45.030049 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029201 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-system-cni-dir\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.030049 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029215 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-host-cni-bin\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.030049 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029231 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-multus-conf-dir\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.030049 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029247 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-host-run-netns\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.030049 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029261 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-host-run-ovn-kubernetes\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.030049 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029275 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-host\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.030049 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029290 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-node-log\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.030049 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029293 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8556c373-2549-4328-aa41-2a05076a088a-cni-binary-copy\") pod \"multus-additional-cni-plugins-v7rlv\" (UID: \"8556c373-2549-4328-aa41-2a05076a088a\") " pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:45.030049 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029307 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/df9fed4f-2388-4d2e-abca-f91fc71334bc-cni-binary-copy\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.030049 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029306 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-etc-sysctl-conf\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.030049 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029352 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpbvn\" (UniqueName: \"kubernetes.io/projected/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-kube-api-access-wpbvn\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.030049 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029367 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a74fe7a2-2170-47f0-aad8-6764d8ab41da-device-dir\") pod \"aws-ebs-csi-driver-node-xjchd\" (UID: \"a74fe7a2-2170-47f0-aad8-6764d8ab41da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" Jul 09 13:29:45.030049 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029379 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8556c373-2549-4328-aa41-2a05076a088a-system-cni-dir\") pod \"multus-additional-cni-plugins-v7rlv\" (UID: \"8556c373-2549-4328-aa41-2a05076a088a\") " pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:45.030049 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029385 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-etc-modprobe-d\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.030049 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029426 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-etc-sysctl-conf\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.030049 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029470 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a74fe7a2-2170-47f0-aad8-6764d8ab41da-registration-dir\") pod \"aws-ebs-csi-driver-node-xjchd\" (UID: \"a74fe7a2-2170-47f0-aad8-6764d8ab41da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" Jul 09 13:29:45.030049 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029489 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-host-run-multus-certs\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.030846 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029529 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-etc-sysctl-d\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.030846 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029563 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a74fe7a2-2170-47f0-aad8-6764d8ab41da-socket-dir\") pod \"aws-ebs-csi-driver-node-xjchd\" (UID: \"a74fe7a2-2170-47f0-aad8-6764d8ab41da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" Jul 09 13:29:45.030846 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029585 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtpfd\" (UniqueName: \"kubernetes.io/projected/df9fed4f-2388-4d2e-abca-f91fc71334bc-kube-api-access-dtpfd\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.030846 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029602 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3b512d09-0288-4fea-864e-fdc734afb1bd-ovn-node-metrics-cert\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.030846 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029623 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-etc-tuned\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.030846 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029639 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a74fe7a2-2170-47f0-aad8-6764d8ab41da-etc-selinux\") pod \"aws-ebs-csi-driver-node-xjchd\" (UID: \"a74fe7a2-2170-47f0-aad8-6764d8ab41da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" Jul 09 13:29:45.030846 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029655 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjqpc\" (UniqueName: \"kubernetes.io/projected/9e8320e4-2106-4edf-b225-d65a4b3eae55-kube-api-access-wjqpc\") pod \"network-check-target-h6qqn\" (UID: \"9e8320e4-2106-4edf-b225-d65a4b3eae55\") " pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:29:45.030846 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029673 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-etc-kubernetes\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.030846 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029694 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-etc-openvswitch\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.030846 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029730 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-sys\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.030846 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029775 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8556c373-2549-4328-aa41-2a05076a088a-cnibin\") pod \"multus-additional-cni-plugins-v7rlv\" (UID: \"8556c373-2549-4328-aa41-2a05076a088a\") " pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:45.030846 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029791 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3b512d09-0288-4fea-864e-fdc734afb1bd-env-overrides\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.030846 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029472 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8556c373-2549-4328-aa41-2a05076a088a-system-cni-dir\") pod \"multus-additional-cni-plugins-v7rlv\" (UID: \"8556c373-2549-4328-aa41-2a05076a088a\") " pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:45.030846 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029852 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a74fe7a2-2170-47f0-aad8-6764d8ab41da-registration-dir\") pod \"aws-ebs-csi-driver-node-xjchd\" (UID: \"a74fe7a2-2170-47f0-aad8-6764d8ab41da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" Jul 09 13:29:45.030846 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029858 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-host-run-multus-certs\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.030846 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029894 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-host-var-lib-cni-multus\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.030846 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029935 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-run\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.031672 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.029970 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-etc-sysctl-d\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.031672 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:45.030011 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 13:29:45.031672 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.030114 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a74fe7a2-2170-47f0-aad8-6764d8ab41da-socket-dir\") pod \"aws-ebs-csi-driver-node-xjchd\" (UID: \"a74fe7a2-2170-47f0-aad8-6764d8ab41da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" Jul 09 13:29:45.031672 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:45.030122 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-original-pull-secret podName:fb20e4df-4233-4fb6-a1f7-e58a789e3d56 nodeName:}" failed. No retries permitted until 2026-07-09 13:29:45.530071531 +0000 UTC m=+3.035373792 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-original-pull-secret") pod "global-pull-secret-syncer-65fnv" (UID: "fb20e4df-4233-4fb6-a1f7-e58a789e3d56") : object "kube-system"/"original-pull-secret" not registered Jul 09 13:29:45.031672 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.030264 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8556c373-2549-4328-aa41-2a05076a088a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v7rlv\" (UID: \"8556c373-2549-4328-aa41-2a05076a088a\") " pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:45.031672 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.030307 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-etc-systemd\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.031672 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.030345 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-multus-conf-dir\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.031672 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.030366 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8556c373-2549-4328-aa41-2a05076a088a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v7rlv\" (UID: \"8556c373-2549-4328-aa41-2a05076a088a\") " pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:45.031672 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.030386 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-host-run-ovn-kubernetes\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.031672 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.030418 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-host\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.031672 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.030777 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/df9fed4f-2388-4d2e-abca-f91fc71334bc-multus-daemon-config\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.031672 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.030885 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a74fe7a2-2170-47f0-aad8-6764d8ab41da-etc-selinux\") pod \"aws-ebs-csi-driver-node-xjchd\" (UID: \"a74fe7a2-2170-47f0-aad8-6764d8ab41da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" Jul 09 13:29:45.031672 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.030934 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-multus-socket-dir-parent\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.031672 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.030968 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-node-log\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.031672 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.031001 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-host-run-netns\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.031672 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.031061 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-host-cni-bin\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.031672 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.031105 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-system-cni-dir\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.032487 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.031345 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-sys\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.032487 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.031394 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-etc-openvswitch\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.032487 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.031438 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8556c373-2549-4328-aa41-2a05076a088a-cnibin\") pod \"multus-additional-cni-plugins-v7rlv\" (UID: \"8556c373-2549-4328-aa41-2a05076a088a\") " pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:45.032487 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.031475 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a74fe7a2-2170-47f0-aad8-6764d8ab41da-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xjchd\" (UID: \"a74fe7a2-2170-47f0-aad8-6764d8ab41da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" Jul 09 13:29:45.032487 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.031501 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-hostroot\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.032487 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.031506 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3b512d09-0288-4fea-864e-fdc734afb1bd-ovnkube-script-lib\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.032487 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.031539 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-etc-sysconfig\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.032487 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.031565 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-dbus\") pod \"global-pull-secret-syncer-65fnv\" (UID: \"fb20e4df-4233-4fb6-a1f7-e58a789e3d56\") " pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:29:45.032487 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.031570 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-etc-kubernetes\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.032487 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.031597 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7fc8a7a4-1f79-475a-9eed-31141361e131-serviceca\") pod \"node-ca-fcwpf\" (UID: \"7fc8a7a4-1f79-475a-9eed-31141361e131\") " pod="openshift-image-registry/node-ca-fcwpf" Jul 09 13:29:45.032487 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.031605 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-hostroot\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.032487 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.031648 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a74fe7a2-2170-47f0-aad8-6764d8ab41da-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xjchd\" (UID: \"a74fe7a2-2170-47f0-aad8-6764d8ab41da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" Jul 09 13:29:45.032487 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.031717 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-etc-sysconfig\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.032487 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.031837 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-dbus\") pod \"global-pull-secret-syncer-65fnv\" (UID: \"fb20e4df-4233-4fb6-a1f7-e58a789e3d56\") " pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:29:45.032487 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.031870 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzn62\" (UniqueName: \"kubernetes.io/projected/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-kube-api-access-wzn62\") pod \"network-metrics-daemon-5zl2x\" (UID: \"fbe44bc5-d23a-4535-b1d8-74ec52d0ce47\") " pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:29:45.032487 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.031916 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-os-release\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.032487 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.031944 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-host-run-k8s-cni-cncf-io\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.032487 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.031981 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-run-systemd\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.033376 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032006 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-host-cni-netd\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.033376 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032041 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7fc8a7a4-1f79-475a-9eed-31141361e131-serviceca\") pod \"node-ca-fcwpf\" (UID: \"7fc8a7a4-1f79-475a-9eed-31141361e131\") " pod="openshift-image-registry/node-ca-fcwpf" Jul 09 13:29:45.033376 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032048 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8556c373-2549-4328-aa41-2a05076a088a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v7rlv\" (UID: \"8556c373-2549-4328-aa41-2a05076a088a\") " pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:45.033376 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032091 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-kubelet-config\") pod \"global-pull-secret-syncer-65fnv\" (UID: \"fb20e4df-4233-4fb6-a1f7-e58a789e3d56\") " pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:29:45.033376 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032116 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a74fe7a2-2170-47f0-aad8-6764d8ab41da-sys-fs\") pod \"aws-ebs-csi-driver-node-xjchd\" (UID: \"a74fe7a2-2170-47f0-aad8-6764d8ab41da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" Jul 09 13:29:45.033376 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032125 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-host-run-k8s-cni-cncf-io\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.033376 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032142 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k247d\" (UniqueName: \"kubernetes.io/projected/a74fe7a2-2170-47f0-aad8-6764d8ab41da-kube-api-access-k247d\") pod \"aws-ebs-csi-driver-node-xjchd\" (UID: \"a74fe7a2-2170-47f0-aad8-6764d8ab41da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" Jul 09 13:29:45.033376 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032172 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-cnibin\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.033376 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032209 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-host-run-netns\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.033376 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032277 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-host-var-lib-kubelet\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.033376 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032295 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-cnibin\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.033376 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032325 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-run-systemd\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.033376 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032337 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c9f5a139-bec5-4097-8ca2-0618cd70bc56-hosts-file\") pod \"node-resolver-k5fs2\" (UID: \"c9f5a139-bec5-4097-8ca2-0618cd70bc56\") " pod="openshift-dns/node-resolver-k5fs2" Jul 09 13:29:45.033376 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032402 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-host-var-lib-kubelet\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.033376 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032390 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-host-run-netns\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.033376 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032420 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c9f5a139-bec5-4097-8ca2-0618cd70bc56-hosts-file\") pod \"node-resolver-k5fs2\" (UID: \"c9f5a139-bec5-4097-8ca2-0618cd70bc56\") " pod="openshift-dns/node-resolver-k5fs2" Jul 09 13:29:45.033376 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032421 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-host-cni-netd\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.033376 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032472 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-kubelet-config\") pod \"global-pull-secret-syncer-65fnv\" (UID: \"fb20e4df-4233-4fb6-a1f7-e58a789e3d56\") " pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:29:45.034276 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032472 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-os-release\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.034276 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032474 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a74fe7a2-2170-47f0-aad8-6764d8ab41da-sys-fs\") pod \"aws-ebs-csi-driver-node-xjchd\" (UID: \"a74fe7a2-2170-47f0-aad8-6764d8ab41da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" Jul 09 13:29:45.034276 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032511 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9f5a139-bec5-4097-8ca2-0618cd70bc56-tmp-dir\") pod \"node-resolver-k5fs2\" (UID: \"c9f5a139-bec5-4097-8ca2-0618cd70bc56\") " pod="openshift-dns/node-resolver-k5fs2" Jul 09 13:29:45.034276 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032594 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-etc-kubernetes\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.034276 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032631 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-metrics-certs\") pod \"network-metrics-daemon-5zl2x\" (UID: \"fbe44bc5-d23a-4535-b1d8-74ec52d0ce47\") " pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:29:45.034276 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032659 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-host-var-lib-cni-bin\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.034276 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032648 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8556c373-2549-4328-aa41-2a05076a088a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v7rlv\" (UID: \"8556c373-2549-4328-aa41-2a05076a088a\") " pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:45.034276 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032704 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-host-kubelet\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.034276 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032731 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-log-socket\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.034276 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032758 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3b512d09-0288-4fea-864e-fdc734afb1bd-ovnkube-config\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.034276 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032759 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-etc-kubernetes\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.034276 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032816 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-log-socket\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.034276 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032820 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8556c373-2549-4328-aa41-2a05076a088a-os-release\") pod \"multus-additional-cni-plugins-v7rlv\" (UID: \"8556c373-2549-4328-aa41-2a05076a088a\") " pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:45.034276 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032859 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-host-var-lib-cni-bin\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.034276 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032915 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8556c373-2549-4328-aa41-2a05076a088a-os-release\") pod \"multus-additional-cni-plugins-v7rlv\" (UID: \"8556c373-2549-4328-aa41-2a05076a088a\") " pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:45.034276 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032973 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7fc8a7a4-1f79-475a-9eed-31141361e131-host\") pod \"node-ca-fcwpf\" (UID: \"7fc8a7a4-1f79-475a-9eed-31141361e131\") " pod="openshift-image-registry/node-ca-fcwpf" Jul 09 13:29:45.034276 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:45.032994 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:29:45.034276 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032992 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-tmp\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.035104 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.032999 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-host-slash\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.035104 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.033092 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7fc8a7a4-1f79-475a-9eed-31141361e131-host\") pod \"node-ca-fcwpf\" (UID: \"7fc8a7a4-1f79-475a-9eed-31141361e131\") " pod="openshift-image-registry/node-ca-fcwpf" Jul 09 13:29:45.035104 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.033135 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-host-kubelet\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.035104 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.033139 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-host-slash\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.035104 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:45.033164 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-metrics-certs podName:fbe44bc5-d23a-4535-b1d8-74ec52d0ce47 nodeName:}" failed. No retries permitted until 2026-07-09 13:29:45.533126401 +0000 UTC m=+3.038428647 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-metrics-certs") pod "network-metrics-daemon-5zl2x" (UID: "fbe44bc5-d23a-4535-b1d8-74ec52d0ce47") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:29:45.035104 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.033187 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.035104 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.033243 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3b512d09-0288-4fea-864e-fdc734afb1bd-ovnkube-config\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.035104 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.033240 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b512d09-0288-4fea-864e-fdc734afb1bd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.035104 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.033280 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-var-lib-kubelet\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.035104 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.033306 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpvdp\" (UniqueName: \"kubernetes.io/projected/7fc8a7a4-1f79-475a-9eed-31141361e131-kube-api-access-vpvdp\") pod \"node-ca-fcwpf\" (UID: \"7fc8a7a4-1f79-475a-9eed-31141361e131\") " pod="openshift-image-registry/node-ca-fcwpf" Jul 09 13:29:45.035104 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.033360 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-var-lib-kubelet\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.035104 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.033400 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-multus-cni-dir\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.035104 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.033600 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/df9fed4f-2388-4d2e-abca-f91fc71334bc-multus-cni-dir\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.035104 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.033748 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9f5a139-bec5-4097-8ca2-0618cd70bc56-tmp-dir\") pod \"node-resolver-k5fs2\" (UID: \"c9f5a139-bec5-4097-8ca2-0618cd70bc56\") " pod="openshift-dns/node-resolver-k5fs2" Jul 09 13:29:45.035104 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.034560 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3b512d09-0288-4fea-864e-fdc734afb1bd-ovn-node-metrics-cert\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.035104 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.034830 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-etc-tuned\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.043647 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.043468 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlzsz\" (UniqueName: \"kubernetes.io/projected/8556c373-2549-4328-aa41-2a05076a088a-kube-api-access-dlzsz\") pod \"multus-additional-cni-plugins-v7rlv\" (UID: \"8556c373-2549-4328-aa41-2a05076a088a\") " pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:45.043796 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.043687 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpbvn\" (UniqueName: \"kubernetes.io/projected/8441e993-2a09-4e7f-bba1-7f0d6f5c28d5-kube-api-access-wpbvn\") pod \"tuned-cmb9f\" (UID: \"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5\") " pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.044759 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:45.044734 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 13:29:45.044871 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:45.044764 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 13:29:45.044871 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:45.044777 2573 projected.go:196] Error preparing data for projected volume kube-api-access-wjqpc for pod openshift-network-diagnostics/network-check-target-h6qqn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:29:45.044983 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:45.044885 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e8320e4-2106-4edf-b225-d65a4b3eae55-kube-api-access-wjqpc podName:9e8320e4-2106-4edf-b225-d65a4b3eae55 nodeName:}" failed. No retries permitted until 2026-07-09 13:29:45.544862515 +0000 UTC m=+3.050164774 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wjqpc" (UniqueName: "kubernetes.io/projected/9e8320e4-2106-4edf-b225-d65a4b3eae55-kube-api-access-wjqpc") pod "network-check-target-h6qqn" (UID: "9e8320e4-2106-4edf-b225-d65a4b3eae55") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:29:45.046435 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.046368 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtpfd\" (UniqueName: \"kubernetes.io/projected/df9fed4f-2388-4d2e-abca-f91fc71334bc-kube-api-access-dtpfd\") pod \"multus-qb95w\" (UID: \"df9fed4f-2388-4d2e-abca-f91fc71334bc\") " pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.047477 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.047353 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmxxn\" (UniqueName: \"kubernetes.io/projected/3b512d09-0288-4fea-864e-fdc734afb1bd-kube-api-access-tmxxn\") pod \"ovnkube-node-vmms8\" (UID: \"3b512d09-0288-4fea-864e-fdc734afb1bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.047477 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.047412 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzn62\" (UniqueName: \"kubernetes.io/projected/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-kube-api-access-wzn62\") pod \"network-metrics-daemon-5zl2x\" (UID: \"fbe44bc5-d23a-4535-b1d8-74ec52d0ce47\") " pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:29:45.047625 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.047602 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gddm\" (UniqueName: \"kubernetes.io/projected/c9f5a139-bec5-4097-8ca2-0618cd70bc56-kube-api-access-9gddm\") pod \"node-resolver-k5fs2\" (UID: \"c9f5a139-bec5-4097-8ca2-0618cd70bc56\") " pod="openshift-dns/node-resolver-k5fs2" Jul 09 13:29:45.048376 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.048243 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k247d\" (UniqueName: \"kubernetes.io/projected/a74fe7a2-2170-47f0-aad8-6764d8ab41da-kube-api-access-k247d\") pod \"aws-ebs-csi-driver-node-xjchd\" (UID: \"a74fe7a2-2170-47f0-aad8-6764d8ab41da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" Jul 09 13:29:45.049457 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.049434 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpvdp\" (UniqueName: \"kubernetes.io/projected/7fc8a7a4-1f79-475a-9eed-31141361e131-kube-api-access-vpvdp\") pod \"node-ca-fcwpf\" (UID: \"7fc8a7a4-1f79-475a-9eed-31141361e131\") " pod="openshift-image-registry/node-ca-fcwpf" Jul 09 13:29:45.126646 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.126611 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xnbzz" Jul 09 13:29:45.137767 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.137734 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tnf9v" Jul 09 13:29:45.139648 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:45.139604 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3cd6f5c_b27b_4312_84d2_1e706cb7af29.slice/crio-29a435a426ffc0a1955ec4598be0fac45516f6b6812de8ef46ce8964050edd8e WatchSource:0}: Error finding container 29a435a426ffc0a1955ec4598be0fac45516f6b6812de8ef46ce8964050edd8e: Status 404 returned error can't find the container with id 29a435a426ffc0a1955ec4598be0fac45516f6b6812de8ef46ce8964050edd8e Jul 09 13:29:45.148184 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:45.148155 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef1861a0_ca40_4753_80c2_7085d1884d07.slice/crio-7693098f8f58ee775d01894fdee2f5f4b2bd1116c2dd0129aab11cfb62f5a425 WatchSource:0}: Error finding container 7693098f8f58ee775d01894fdee2f5f4b2bd1116c2dd0129aab11cfb62f5a425: Status 404 returned error can't find the container with id 7693098f8f58ee775d01894fdee2f5f4b2bd1116c2dd0129aab11cfb62f5a425 Jul 09 13:29:45.150599 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.150581 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qb95w" Jul 09 13:29:45.158187 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.158162 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:29:45.162268 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:45.162236 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf9fed4f_2388_4d2e_abca_f91fc71334bc.slice/crio-51e2b06165d1071ee8a7ae5b6d09c2d478f0e965940edc0203d9b5b3d1f9d676 WatchSource:0}: Error finding container 51e2b06165d1071ee8a7ae5b6d09c2d478f0e965940edc0203d9b5b3d1f9d676: Status 404 returned error can't find the container with id 51e2b06165d1071ee8a7ae5b6d09c2d478f0e965940edc0203d9b5b3d1f9d676 Jul 09 13:29:45.167423 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.167399 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k5fs2" Jul 09 13:29:45.171109 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:45.171082 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b512d09_0288_4fea_864e_fdc734afb1bd.slice/crio-f829a51b7f54eaccabaaec0a9e02115dca2b6990da4c688bd7397fa7928d9796 WatchSource:0}: Error finding container f829a51b7f54eaccabaaec0a9e02115dca2b6990da4c688bd7397fa7928d9796: Status 404 returned error can't find the container with id f829a51b7f54eaccabaaec0a9e02115dca2b6990da4c688bd7397fa7928d9796 Jul 09 13:29:45.174978 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.174789 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fcwpf" Jul 09 13:29:45.180461 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:45.180432 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9f5a139_bec5_4097_8ca2_0618cd70bc56.slice/crio-41fe2e1a65becc8385c274dbc4ddcb0edce0e78e964f097524d20c66839f592a WatchSource:0}: Error finding container 41fe2e1a65becc8385c274dbc4ddcb0edce0e78e964f097524d20c66839f592a: Status 404 returned error can't find the container with id 41fe2e1a65becc8385c274dbc4ddcb0edce0e78e964f097524d20c66839f592a Jul 09 13:29:45.183007 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.182939 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" Jul 09 13:29:45.186890 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:45.186859 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fc8a7a4_1f79_475a_9eed_31141361e131.slice/crio-4cbc79cc7e1ef8e7b6027d7824eeaffc3e0c97f5f18ebb81f98eb207290a24af WatchSource:0}: Error finding container 4cbc79cc7e1ef8e7b6027d7824eeaffc3e0c97f5f18ebb81f98eb207290a24af: Status 404 returned error can't find the container with id 4cbc79cc7e1ef8e7b6027d7824eeaffc3e0c97f5f18ebb81f98eb207290a24af Jul 09 13:29:45.190405 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.190309 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v7rlv" Jul 09 13:29:45.194653 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:45.194625 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8441e993_2a09_4e7f_bba1_7f0d6f5c28d5.slice/crio-d38eaa13064b2f5c9257e4953475ba6e936936707e4118e9d2c151938b35dc86 WatchSource:0}: Error finding container d38eaa13064b2f5c9257e4953475ba6e936936707e4118e9d2c151938b35dc86: Status 404 returned error can't find the container with id d38eaa13064b2f5c9257e4953475ba6e936936707e4118e9d2c151938b35dc86 Jul 09 13:29:45.198958 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.198938 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" Jul 09 13:29:45.200930 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:45.200906 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8556c373_2549_4328_aa41_2a05076a088a.slice/crio-db76326febdb137088557651b2d5bd46d5066050a1ac50850701de19c98cf48f WatchSource:0}: Error finding container db76326febdb137088557651b2d5bd46d5066050a1ac50850701de19c98cf48f: Status 404 returned error can't find the container with id db76326febdb137088557651b2d5bd46d5066050a1ac50850701de19c98cf48f Jul 09 13:29:45.536730 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.536699 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-metrics-certs\") pod \"network-metrics-daemon-5zl2x\" (UID: \"fbe44bc5-d23a-4535-b1d8-74ec52d0ce47\") " pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:29:45.536901 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.536743 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-original-pull-secret\") pod \"global-pull-secret-syncer-65fnv\" (UID: \"fb20e4df-4233-4fb6-a1f7-e58a789e3d56\") " pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:29:45.536901 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:45.536851 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 13:29:45.536901 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:45.536865 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:29:45.536901 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:45.536901 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-original-pull-secret podName:fb20e4df-4233-4fb6-a1f7-e58a789e3d56 nodeName:}" failed. No retries permitted until 2026-07-09 13:29:46.536888912 +0000 UTC m=+4.042191158 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-original-pull-secret") pod "global-pull-secret-syncer-65fnv" (UID: "fb20e4df-4233-4fb6-a1f7-e58a789e3d56") : object "kube-system"/"original-pull-secret" not registered Jul 09 13:29:45.537069 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:45.536920 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-metrics-certs podName:fbe44bc5-d23a-4535-b1d8-74ec52d0ce47 nodeName:}" failed. No retries permitted until 2026-07-09 13:29:46.536911412 +0000 UTC m=+4.042213661 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-metrics-certs") pod "network-metrics-daemon-5zl2x" (UID: "fbe44bc5-d23a-4535-b1d8-74ec52d0ce47") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:29:45.637941 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.637904 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjqpc\" (UniqueName: \"kubernetes.io/projected/9e8320e4-2106-4edf-b225-d65a4b3eae55-kube-api-access-wjqpc\") pod \"network-check-target-h6qqn\" (UID: \"9e8320e4-2106-4edf-b225-d65a4b3eae55\") " pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:29:45.638131 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:45.638065 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 13:29:45.638131 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:45.638085 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 13:29:45.638131 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:45.638106 2573 projected.go:196] Error preparing data for projected volume kube-api-access-wjqpc for pod openshift-network-diagnostics/network-check-target-h6qqn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:29:45.638268 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:45.638166 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e8320e4-2106-4edf-b225-d65a4b3eae55-kube-api-access-wjqpc podName:9e8320e4-2106-4edf-b225-d65a4b3eae55 nodeName:}" failed. No retries permitted until 2026-07-09 13:29:46.638149235 +0000 UTC m=+4.143451484 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wjqpc" (UniqueName: "kubernetes.io/projected/9e8320e4-2106-4edf-b225-d65a4b3eae55-kube-api-access-wjqpc") pod "network-check-target-h6qqn" (UID: "9e8320e4-2106-4edf-b225-d65a4b3eae55") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:29:45.872776 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.872684 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 13:24:43 +0000 UTC" deadline="2028-04-13 17:17:52.277992402 +0000 UTC" Jul 09 13:29:45.872776 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.872731 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15459h48m6.405263871s" Jul 09 13:29:45.952090 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.952043 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" event={"ID":"3b512d09-0288-4fea-864e-fdc734afb1bd","Type":"ContainerStarted","Data":"f829a51b7f54eaccabaaec0a9e02115dca2b6990da4c688bd7397fa7928d9796"} Jul 09 13:29:45.953373 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.953339 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v7rlv" event={"ID":"8556c373-2549-4328-aa41-2a05076a088a","Type":"ContainerStarted","Data":"db76326febdb137088557651b2d5bd46d5066050a1ac50850701de19c98cf48f"} Jul 09 13:29:45.954670 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.954645 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fcwpf" event={"ID":"7fc8a7a4-1f79-475a-9eed-31141361e131","Type":"ContainerStarted","Data":"4cbc79cc7e1ef8e7b6027d7824eeaffc3e0c97f5f18ebb81f98eb207290a24af"} Jul 09 13:29:45.955920 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.955895 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qb95w" event={"ID":"df9fed4f-2388-4d2e-abca-f91fc71334bc","Type":"ContainerStarted","Data":"51e2b06165d1071ee8a7ae5b6d09c2d478f0e965940edc0203d9b5b3d1f9d676"} Jul 09 13:29:45.957156 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.957134 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tnf9v" event={"ID":"ef1861a0-ca40-4753-80c2-7085d1884d07","Type":"ContainerStarted","Data":"7693098f8f58ee775d01894fdee2f5f4b2bd1116c2dd0129aab11cfb62f5a425"} Jul 09 13:29:45.958270 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.958241 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xnbzz" event={"ID":"b3cd6f5c-b27b-4312-84d2-1e706cb7af29","Type":"ContainerStarted","Data":"29a435a426ffc0a1955ec4598be0fac45516f6b6812de8ef46ce8964050edd8e"} Jul 09 13:29:45.959372 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.959349 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" event={"ID":"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5","Type":"ContainerStarted","Data":"d38eaa13064b2f5c9257e4953475ba6e936936707e4118e9d2c151938b35dc86"} Jul 09 13:29:45.960475 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:45.960455 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k5fs2" event={"ID":"c9f5a139-bec5-4097-8ca2-0618cd70bc56","Type":"ContainerStarted","Data":"41fe2e1a65becc8385c274dbc4ddcb0edce0e78e964f097524d20c66839f592a"} Jul 09 13:29:46.275546 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:29:46.275511 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda74fe7a2_2170_47f0_aad8_6764d8ab41da.slice/crio-79fe58a69c71a35a7f79854815679ba25ddfce6ef8db80a743faa5ba9b67e7b8 WatchSource:0}: Error finding container 79fe58a69c71a35a7f79854815679ba25ddfce6ef8db80a743faa5ba9b67e7b8: Status 404 returned error can't find the container with id 79fe58a69c71a35a7f79854815679ba25ddfce6ef8db80a743faa5ba9b67e7b8 Jul 09 13:29:46.546113 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:46.545767 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-metrics-certs\") pod \"network-metrics-daemon-5zl2x\" (UID: \"fbe44bc5-d23a-4535-b1d8-74ec52d0ce47\") " pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:29:46.546113 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:46.546072 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-original-pull-secret\") pod \"global-pull-secret-syncer-65fnv\" (UID: \"fb20e4df-4233-4fb6-a1f7-e58a789e3d56\") " pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:29:46.546330 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:46.546228 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 13:29:46.546330 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:46.546291 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-original-pull-secret podName:fb20e4df-4233-4fb6-a1f7-e58a789e3d56 nodeName:}" failed. No retries permitted until 2026-07-09 13:29:48.546273065 +0000 UTC m=+6.051575313 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-original-pull-secret") pod "global-pull-secret-syncer-65fnv" (UID: "fb20e4df-4233-4fb6-a1f7-e58a789e3d56") : object "kube-system"/"original-pull-secret" not registered Jul 09 13:29:46.546444 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:46.546359 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:29:46.546444 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:46.546394 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-metrics-certs podName:fbe44bc5-d23a-4535-b1d8-74ec52d0ce47 nodeName:}" failed. No retries permitted until 2026-07-09 13:29:48.546382575 +0000 UTC m=+6.051684825 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-metrics-certs") pod "network-metrics-daemon-5zl2x" (UID: "fbe44bc5-d23a-4535-b1d8-74ec52d0ce47") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:29:46.647429 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:46.647392 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjqpc\" (UniqueName: \"kubernetes.io/projected/9e8320e4-2106-4edf-b225-d65a4b3eae55-kube-api-access-wjqpc\") pod \"network-check-target-h6qqn\" (UID: \"9e8320e4-2106-4edf-b225-d65a4b3eae55\") " pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:29:46.647604 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:46.647545 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 13:29:46.647604 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:46.647564 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 13:29:46.647604 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:46.647576 2573 projected.go:196] Error preparing data for projected volume kube-api-access-wjqpc for pod openshift-network-diagnostics/network-check-target-h6qqn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:29:46.647758 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:46.647632 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e8320e4-2106-4edf-b225-d65a4b3eae55-kube-api-access-wjqpc podName:9e8320e4-2106-4edf-b225-d65a4b3eae55 nodeName:}" failed. No retries permitted until 2026-07-09 13:29:48.647614341 +0000 UTC m=+6.152916604 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wjqpc" (UniqueName: "kubernetes.io/projected/9e8320e4-2106-4edf-b225-d65a4b3eae55-kube-api-access-wjqpc") pod "network-check-target-h6qqn" (UID: "9e8320e4-2106-4edf-b225-d65a4b3eae55") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:29:46.650657 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:46.650621 2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 13:29:46.942220 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:46.942190 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:29:46.942655 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:46.942320 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65fnv" podUID="fb20e4df-4233-4fb6-a1f7-e58a789e3d56" Jul 09 13:29:46.942737 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:46.942700 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:29:46.942830 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:46.942808 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5zl2x" podUID="fbe44bc5-d23a-4535-b1d8-74ec52d0ce47" Jul 09 13:29:46.942896 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:46.942874 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:29:46.942954 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:46.942941 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6qqn" podUID="9e8320e4-2106-4edf-b225-d65a4b3eae55" Jul 09 13:29:46.982766 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:46.982724 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-87.ec2.internal" event={"ID":"dc8175f5b3169ac412d470dea8cf3d3d","Type":"ContainerStarted","Data":"6a911124da016c94d58aa96d1991bcaabda06d2f23c44e9df06bf744f6a1490e"} Jul 09 13:29:46.986565 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:46.986482 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" event={"ID":"a74fe7a2-2170-47f0-aad8-6764d8ab41da","Type":"ContainerStarted","Data":"79fe58a69c71a35a7f79854815679ba25ddfce6ef8db80a743faa5ba9b67e7b8"} Jul 09 13:29:47.002417 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:47.002340 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-87.ec2.internal" podStartSLOduration=4.002319242 podStartE2EDuration="4.002319242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:29:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:29:47.001946321 +0000 UTC m=+4.507248590" watchObservedRunningTime="2026-07-09 13:29:47.002319242 +0000 UTC m=+4.507621511" Jul 09 13:29:48.019727 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:48.019661 2573 generic.go:356] "Generic (PLEG): container finished" podID="a3649323ce8714df59b0d7e58d8cfecb" containerID="8917ee689f2c5466e50eb52bccab3cc498d9ed8f25ee29920650b91ce2f8c77e" exitCode=0 Jul 09 13:29:48.020765 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:48.020709 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-87.ec2.internal" event={"ID":"a3649323ce8714df59b0d7e58d8cfecb","Type":"ContainerDied","Data":"8917ee689f2c5466e50eb52bccab3cc498d9ed8f25ee29920650b91ce2f8c77e"} Jul 09 13:29:48.564654 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:48.564607 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-metrics-certs\") pod \"network-metrics-daemon-5zl2x\" (UID: \"fbe44bc5-d23a-4535-b1d8-74ec52d0ce47\") " pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:29:48.564853 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:48.564679 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-original-pull-secret\") pod \"global-pull-secret-syncer-65fnv\" (UID: \"fb20e4df-4233-4fb6-a1f7-e58a789e3d56\") " pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:29:48.564853 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:48.564806 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 13:29:48.564991 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:48.564863 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-original-pull-secret podName:fb20e4df-4233-4fb6-a1f7-e58a789e3d56 nodeName:}" failed. No retries permitted until 2026-07-09 13:29:52.564846575 +0000 UTC m=+10.070148826 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-original-pull-secret") pod "global-pull-secret-syncer-65fnv" (UID: "fb20e4df-4233-4fb6-a1f7-e58a789e3d56") : object "kube-system"/"original-pull-secret" not registered Jul 09 13:29:48.565347 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:48.565325 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:29:48.565438 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:48.565387 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-metrics-certs podName:fbe44bc5-d23a-4535-b1d8-74ec52d0ce47 nodeName:}" failed. No retries permitted until 2026-07-09 13:29:52.565370185 +0000 UTC m=+10.070672434 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-metrics-certs") pod "network-metrics-daemon-5zl2x" (UID: "fbe44bc5-d23a-4535-b1d8-74ec52d0ce47") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:29:48.666439 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:48.666397 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjqpc\" (UniqueName: \"kubernetes.io/projected/9e8320e4-2106-4edf-b225-d65a4b3eae55-kube-api-access-wjqpc\") pod \"network-check-target-h6qqn\" (UID: \"9e8320e4-2106-4edf-b225-d65a4b3eae55\") " pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:29:48.666628 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:48.666566 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 13:29:48.666628 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:48.666585 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 13:29:48.666628 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:48.666597 2573 projected.go:196] Error preparing data for projected volume kube-api-access-wjqpc for pod openshift-network-diagnostics/network-check-target-h6qqn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:29:48.666785 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:48.666664 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e8320e4-2106-4edf-b225-d65a4b3eae55-kube-api-access-wjqpc podName:9e8320e4-2106-4edf-b225-d65a4b3eae55 nodeName:}" failed. No retries permitted until 2026-07-09 13:29:52.666645871 +0000 UTC m=+10.171948126 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wjqpc" (UniqueName: "kubernetes.io/projected/9e8320e4-2106-4edf-b225-d65a4b3eae55-kube-api-access-wjqpc") pod "network-check-target-h6qqn" (UID: "9e8320e4-2106-4edf-b225-d65a4b3eae55") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:29:48.941052 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:48.941004 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:29:48.941244 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:48.941166 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5zl2x" podUID="fbe44bc5-d23a-4535-b1d8-74ec52d0ce47" Jul 09 13:29:48.941244 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:48.941220 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:29:48.941349 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:48.941324 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6qqn" podUID="9e8320e4-2106-4edf-b225-d65a4b3eae55" Jul 09 13:29:48.941705 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:48.941459 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:29:48.941705 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:48.941545 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65fnv" podUID="fb20e4df-4233-4fb6-a1f7-e58a789e3d56" Jul 09 13:29:50.939894 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:50.939861 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:29:50.939894 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:50.939906 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:29:50.939894 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:50.939996 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6qqn" podUID="9e8320e4-2106-4edf-b225-d65a4b3eae55" Jul 09 13:29:50.939894 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:50.940120 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5zl2x" podUID="fbe44bc5-d23a-4535-b1d8-74ec52d0ce47" Jul 09 13:29:50.939894 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:50.940210 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:29:50.939894 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:50.940334 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65fnv" podUID="fb20e4df-4233-4fb6-a1f7-e58a789e3d56" Jul 09 13:29:52.604063 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:52.604000 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-metrics-certs\") pod \"network-metrics-daemon-5zl2x\" (UID: \"fbe44bc5-d23a-4535-b1d8-74ec52d0ce47\") " pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:29:52.604543 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:52.604086 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-original-pull-secret\") pod \"global-pull-secret-syncer-65fnv\" (UID: \"fb20e4df-4233-4fb6-a1f7-e58a789e3d56\") " pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:29:52.604543 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:52.604202 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:29:52.604543 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:52.604242 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 13:29:52.604543 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:52.604280 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-metrics-certs podName:fbe44bc5-d23a-4535-b1d8-74ec52d0ce47 nodeName:}" failed. No retries permitted until 2026-07-09 13:30:00.604259877 +0000 UTC m=+18.109562131 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-metrics-certs") pod "network-metrics-daemon-5zl2x" (UID: "fbe44bc5-d23a-4535-b1d8-74ec52d0ce47") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:29:52.604543 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:52.604305 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-original-pull-secret podName:fb20e4df-4233-4fb6-a1f7-e58a789e3d56 nodeName:}" failed. No retries permitted until 2026-07-09 13:30:00.604287637 +0000 UTC m=+18.109589897 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-original-pull-secret") pod "global-pull-secret-syncer-65fnv" (UID: "fb20e4df-4233-4fb6-a1f7-e58a789e3d56") : object "kube-system"/"original-pull-secret" not registered Jul 09 13:29:52.705000 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:52.704959 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjqpc\" (UniqueName: \"kubernetes.io/projected/9e8320e4-2106-4edf-b225-d65a4b3eae55-kube-api-access-wjqpc\") pod \"network-check-target-h6qqn\" (UID: \"9e8320e4-2106-4edf-b225-d65a4b3eae55\") " pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:29:52.705195 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:52.705144 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 13:29:52.705195 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:52.705166 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 13:29:52.705195 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:52.705178 2573 projected.go:196] Error preparing data for projected volume kube-api-access-wjqpc for pod openshift-network-diagnostics/network-check-target-h6qqn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:29:52.705347 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:52.705248 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e8320e4-2106-4edf-b225-d65a4b3eae55-kube-api-access-wjqpc podName:9e8320e4-2106-4edf-b225-d65a4b3eae55 nodeName:}" failed. No retries permitted until 2026-07-09 13:30:00.705226852 +0000 UTC m=+18.210529114 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wjqpc" (UniqueName: "kubernetes.io/projected/9e8320e4-2106-4edf-b225-d65a4b3eae55-kube-api-access-wjqpc") pod "network-check-target-h6qqn" (UID: "9e8320e4-2106-4edf-b225-d65a4b3eae55") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:29:52.940911 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:52.940875 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:29:52.941099 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:52.940978 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6qqn" podUID="9e8320e4-2106-4edf-b225-d65a4b3eae55" Jul 09 13:29:52.941179 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:52.941153 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:29:52.941308 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:52.941153 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:29:52.941412 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:52.941271 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65fnv" podUID="fb20e4df-4233-4fb6-a1f7-e58a789e3d56" Jul 09 13:29:52.941719 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:52.941643 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5zl2x" podUID="fbe44bc5-d23a-4535-b1d8-74ec52d0ce47" Jul 09 13:29:54.940534 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:54.940499 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:29:54.940988 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:54.940512 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:29:54.940988 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:54.940627 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65fnv" podUID="fb20e4df-4233-4fb6-a1f7-e58a789e3d56" Jul 09 13:29:54.940988 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:54.940640 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:29:54.940988 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:54.940711 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5zl2x" podUID="fbe44bc5-d23a-4535-b1d8-74ec52d0ce47" Jul 09 13:29:54.940988 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:54.940776 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6qqn" podUID="9e8320e4-2106-4edf-b225-d65a4b3eae55" Jul 09 13:29:56.939710 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:56.939678 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:29:56.940169 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:56.939682 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:29:56.940169 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:56.939788 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6qqn" podUID="9e8320e4-2106-4edf-b225-d65a4b3eae55" Jul 09 13:29:56.940169 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:56.939866 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65fnv" podUID="fb20e4df-4233-4fb6-a1f7-e58a789e3d56" Jul 09 13:29:56.940169 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:56.939689 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:29:56.940169 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:56.939966 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5zl2x" podUID="fbe44bc5-d23a-4535-b1d8-74ec52d0ce47" Jul 09 13:29:58.939583 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:58.939543 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:29:58.940084 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:58.939543 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:29:58.940084 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:58.939680 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65fnv" podUID="fb20e4df-4233-4fb6-a1f7-e58a789e3d56" Jul 09 13:29:58.940084 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:58.939758 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5zl2x" podUID="fbe44bc5-d23a-4535-b1d8-74ec52d0ce47" Jul 09 13:29:58.940084 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:29:58.939548 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:29:58.940084 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:29:58.939833 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6qqn" podUID="9e8320e4-2106-4edf-b225-d65a4b3eae55" Jul 09 13:30:00.667096 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:00.667059 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-original-pull-secret\") pod \"global-pull-secret-syncer-65fnv\" (UID: \"fb20e4df-4233-4fb6-a1f7-e58a789e3d56\") " pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:30:00.667623 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:00.667142 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-metrics-certs\") pod \"network-metrics-daemon-5zl2x\" (UID: \"fbe44bc5-d23a-4535-b1d8-74ec52d0ce47\") " pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:30:00.667623 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:00.667238 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:30:00.667623 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:00.667241 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 13:30:00.667623 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:00.667305 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-metrics-certs podName:fbe44bc5-d23a-4535-b1d8-74ec52d0ce47 nodeName:}" failed. No retries permitted until 2026-07-09 13:30:16.66728968 +0000 UTC m=+34.172591928 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-metrics-certs") pod "network-metrics-daemon-5zl2x" (UID: "fbe44bc5-d23a-4535-b1d8-74ec52d0ce47") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:30:00.667623 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:00.667319 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-original-pull-secret podName:fb20e4df-4233-4fb6-a1f7-e58a789e3d56 nodeName:}" failed. No retries permitted until 2026-07-09 13:30:16.667312883 +0000 UTC m=+34.172615128 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-original-pull-secret") pod "global-pull-secret-syncer-65fnv" (UID: "fb20e4df-4233-4fb6-a1f7-e58a789e3d56") : object "kube-system"/"original-pull-secret" not registered Jul 09 13:30:00.768157 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:00.768121 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjqpc\" (UniqueName: \"kubernetes.io/projected/9e8320e4-2106-4edf-b225-d65a4b3eae55-kube-api-access-wjqpc\") pod \"network-check-target-h6qqn\" (UID: \"9e8320e4-2106-4edf-b225-d65a4b3eae55\") " pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:30:00.768379 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:00.768356 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 13:30:00.768446 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:00.768385 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 13:30:00.768446 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:00.768400 2573 projected.go:196] Error preparing data for projected volume kube-api-access-wjqpc for pod openshift-network-diagnostics/network-check-target-h6qqn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:30:00.768511 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:00.768465 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e8320e4-2106-4edf-b225-d65a4b3eae55-kube-api-access-wjqpc podName:9e8320e4-2106-4edf-b225-d65a4b3eae55 nodeName:}" failed. No retries permitted until 2026-07-09 13:30:16.768446585 +0000 UTC m=+34.273748837 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wjqpc" (UniqueName: "kubernetes.io/projected/9e8320e4-2106-4edf-b225-d65a4b3eae55-kube-api-access-wjqpc") pod "network-check-target-h6qqn" (UID: "9e8320e4-2106-4edf-b225-d65a4b3eae55") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:30:00.940334 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:00.940254 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:30:00.940334 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:00.940291 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:30:00.940531 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:00.940254 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:30:00.940531 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:00.940399 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5zl2x" podUID="fbe44bc5-d23a-4535-b1d8-74ec52d0ce47" Jul 09 13:30:00.940531 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:00.940480 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65fnv" podUID="fb20e4df-4233-4fb6-a1f7-e58a789e3d56" Jul 09 13:30:00.940652 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:00.940561 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6qqn" podUID="9e8320e4-2106-4edf-b225-d65a4b3eae55" Jul 09 13:30:02.942959 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:02.942927 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:30:02.942959 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:02.942965 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:30:02.943498 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:02.942965 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:30:02.943498 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:02.943073 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5zl2x" podUID="fbe44bc5-d23a-4535-b1d8-74ec52d0ce47" Jul 09 13:30:02.943498 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:02.943156 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65fnv" podUID="fb20e4df-4233-4fb6-a1f7-e58a789e3d56" Jul 09 13:30:02.943498 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:02.943222 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6qqn" podUID="9e8320e4-2106-4edf-b225-d65a4b3eae55" Jul 09 13:30:04.943136 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:04.942952 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:30:04.943876 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:04.942954 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:30:04.943876 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:04.943233 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65fnv" podUID="fb20e4df-4233-4fb6-a1f7-e58a789e3d56" Jul 09 13:30:04.943876 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:04.942954 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:30:04.943876 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:04.943349 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5zl2x" podUID="fbe44bc5-d23a-4535-b1d8-74ec52d0ce47" Jul 09 13:30:04.943876 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:04.943382 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6qqn" podUID="9e8320e4-2106-4edf-b225-d65a4b3eae55" Jul 09 13:30:05.055573 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:05.055542 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" event={"ID":"8441e993-2a09-4e7f-bba1-7f0d6f5c28d5","Type":"ContainerStarted","Data":"5bcc06715188da54025e93308892cd90d3094b0d4f833dcf9e665a0e325b2fdb"} Jul 09 13:30:05.057154 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:05.057126 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k5fs2" event={"ID":"c9f5a139-bec5-4097-8ca2-0618cd70bc56","Type":"ContainerStarted","Data":"756dff4dcdcc4c2c974530cbb640f4de63af29f3d45ed469b3baadf6dce3ddd3"} Jul 09 13:30:05.058656 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:05.058628 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" event={"ID":"3b512d09-0288-4fea-864e-fdc734afb1bd","Type":"ContainerStarted","Data":"3c46e25297e855e93f79e16f9b940d26ace07642e8643d1ca59680cb463166d1"} Jul 09 13:30:05.060131 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:05.060030 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" event={"ID":"a74fe7a2-2170-47f0-aad8-6764d8ab41da","Type":"ContainerStarted","Data":"34d8cd22a147f3ff0a77f769242c3202424117cd648fae338cef5902c078548a"} Jul 09 13:30:05.062096 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:05.062062 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-87.ec2.internal" event={"ID":"a3649323ce8714df59b0d7e58d8cfecb","Type":"ContainerStarted","Data":"86975c2226629c2a67d62d0422e2042ef4e6e249994d53b4ac9b44f1e7b1a859"} Jul 09 13:30:05.063617 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:05.063595 2573 generic.go:356] "Generic (PLEG): container finished" podID="8556c373-2549-4328-aa41-2a05076a088a" containerID="888555dd0f83858e62df55b2671eddece879a20b126bf46d44e3ef202c2f9346" exitCode=0 Jul 09 13:30:05.063713 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:05.063662 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v7rlv" event={"ID":"8556c373-2549-4328-aa41-2a05076a088a","Type":"ContainerDied","Data":"888555dd0f83858e62df55b2671eddece879a20b126bf46d44e3ef202c2f9346"} Jul 09 13:30:05.065122 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:05.065090 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fcwpf" event={"ID":"7fc8a7a4-1f79-475a-9eed-31141361e131","Type":"ContainerStarted","Data":"b7da294f6d065d857ad8d36705eabdd7f841ac1cbdcfc4dd43b8482f90d636e5"} Jul 09 13:30:05.066695 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:05.066673 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qb95w" event={"ID":"df9fed4f-2388-4d2e-abca-f91fc71334bc","Type":"ContainerStarted","Data":"ea9825c9589bf7f04c3e65ca467011eae54c4c842067a92541c0720c59a01d33"} Jul 09 13:30:05.068041 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:05.067998 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tnf9v" event={"ID":"ef1861a0-ca40-4753-80c2-7085d1884d07","Type":"ContainerStarted","Data":"a39a9a7c7041e92b046e64ce5468235269a5b8a02b1cd1dc2bd73a9528242d03"} Jul 09 13:30:05.080766 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:05.080723 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-cmb9f" podStartSLOduration=2.7286480920000002 podStartE2EDuration="22.080709141s" totalImagesPullingTime="19.352061049s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:29:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:05.079949944 +0000 UTC m=+22.585252214" watchObservedRunningTime="2026-07-09 13:30:05.080709141 +0000 UTC m=+22.586011409" Jul 09 13:30:05.102793 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:05.102745 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-87.ec2.internal" podStartSLOduration=22.102729011 podStartE2EDuration="22.102729011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:29:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:05.102522547 +0000 UTC m=+22.607824815" watchObservedRunningTime="2026-07-09 13:30:05.102729011 +0000 UTC m=+22.608031278" Jul 09 13:30:05.145517 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:05.145278 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qb95w" podStartSLOduration=2.74774724 podStartE2EDuration="22.145257148s" totalImagesPullingTime="19.397509908s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:29:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:05.14514368 +0000 UTC m=+22.650445948" watchObservedRunningTime="2026-07-09 13:30:05.145257148 +0000 UTC m=+22.650559419" Jul 09 13:30:05.145607 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:05.145573 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tnf9v" podStartSLOduration=3.766104692 podStartE2EDuration="23.145567569s" totalImagesPullingTime="19.379462877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:29:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:05.123489534 +0000 UTC m=+22.628791795" watchObservedRunningTime="2026-07-09 13:30:05.145567569 +0000 UTC m=+22.650869836" Jul 09 13:30:05.166351 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:05.166177 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-k5fs2" podStartSLOduration=7.533539111 podStartE2EDuration="22.166159283s" totalImagesPullingTime="14.632620172s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:29:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:05.166057428 +0000 UTC m=+22.671359714" watchObservedRunningTime="2026-07-09 13:30:05.166159283 +0000 UTC m=+22.671461550" Jul 09 13:30:05.182466 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:05.182357 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fcwpf" podStartSLOduration=2.824870252 podStartE2EDuration="22.182341823s" totalImagesPullingTime="19.357471571s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:29:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:05.181545709 +0000 UTC m=+22.686847976" watchObservedRunningTime="2026-07-09 13:30:05.182341823 +0000 UTC m=+22.687644091" Jul 09 13:30:05.602960 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:05.602922 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tnf9v" Jul 09 13:30:05.603815 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:05.603795 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tnf9v" Jul 09 13:30:06.072624 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:06.072544 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xnbzz" event={"ID":"b3cd6f5c-b27b-4312-84d2-1e706cb7af29","Type":"ContainerStarted","Data":"cc53b1a17cec9935b7b8c38e3199091b3ca6b0dd6483836dc670ed24ae123480"} Jul 09 13:30:06.075972 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:06.075949 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmms8_3b512d09-0288-4fea-864e-fdc734afb1bd/ovn-acl-logging/0.log" Jul 09 13:30:06.076346 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:06.076316 2573 generic.go:356] "Generic (PLEG): container finished" podID="3b512d09-0288-4fea-864e-fdc734afb1bd" containerID="c0faf4dad177e85fead341499327e7ef4cdbcfaf5abb7505722489bb82afda19" exitCode=1 Jul 09 13:30:06.076490 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:06.076453 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" event={"ID":"3b512d09-0288-4fea-864e-fdc734afb1bd","Type":"ContainerStarted","Data":"5abf294f50154fab774d2398fc57da253c3b66c613e99549f8b5e3d78245437d"} Jul 09 13:30:06.076561 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:06.076540 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" event={"ID":"3b512d09-0288-4fea-864e-fdc734afb1bd","Type":"ContainerStarted","Data":"4358cda201f6fac1b856453b0e5329596a2df2625a15b150fd36b7c10a183f2a"} Jul 09 13:30:06.076666 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:06.076603 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" event={"ID":"3b512d09-0288-4fea-864e-fdc734afb1bd","Type":"ContainerStarted","Data":"3aa84649d7b207618e2e1698e78d5cabfdc3266d81f898b80dccc4e15873dd3b"} Jul 09 13:30:06.076666 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:06.076655 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" event={"ID":"3b512d09-0288-4fea-864e-fdc734afb1bd","Type":"ContainerStarted","Data":"db4a5d32da6034749d3cf9b5c2cff32910db8a9c5d681752cc0562d5c2eea4b4"} Jul 09 13:30:06.076787 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:06.076706 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" event={"ID":"3b512d09-0288-4fea-864e-fdc734afb1bd","Type":"ContainerDied","Data":"c0faf4dad177e85fead341499327e7ef4cdbcfaf5abb7505722489bb82afda19"} Jul 09 13:30:06.077499 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:06.077473 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tnf9v" Jul 09 13:30:06.077627 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:06.077537 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tnf9v" Jul 09 13:30:06.109565 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:06.109506 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xnbzz" podStartSLOduration=5.025384795 podStartE2EDuration="24.10948836s" totalImagesPullingTime="19.084103565s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:29:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:06.091407945 +0000 UTC m=+23.596710250" watchObservedRunningTime="2026-07-09 13:30:06.10948836 +0000 UTC m=+23.614790627" Jul 09 13:30:06.485305 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:06.485278 2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 13:30:06.888696 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:06.888591 2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T13:30:06.485297968Z","UUID":"7acbb52c-04e2-43ee-88d8-0ba7ca82dafd","Handler":null,"Name":"","Endpoint":""} Jul 09 13:30:06.890409 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:06.890387 2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 13:30:06.890542 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:06.890424 2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 13:30:06.940294 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:06.940253 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:30:06.940538 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:06.940253 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:30:06.940538 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:06.940382 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6qqn" podUID="9e8320e4-2106-4edf-b225-d65a4b3eae55" Jul 09 13:30:06.940538 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:06.940484 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:30:06.940699 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:06.940484 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5zl2x" podUID="fbe44bc5-d23a-4535-b1d8-74ec52d0ce47" Jul 09 13:30:06.940699 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:06.940572 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65fnv" podUID="fb20e4df-4233-4fb6-a1f7-e58a789e3d56" Jul 09 13:30:07.081864 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:07.081826 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" event={"ID":"a74fe7a2-2170-47f0-aad8-6764d8ab41da","Type":"ContainerStarted","Data":"65b9c56b1d75fd772c6e1ec973aa56c6113db7ccc8145e0e94ef540dd453b3de"} Jul 09 13:30:08.087307 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:08.087139 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmms8_3b512d09-0288-4fea-864e-fdc734afb1bd/ovn-acl-logging/0.log" Jul 09 13:30:08.087741 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:08.087715 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" event={"ID":"3b512d09-0288-4fea-864e-fdc734afb1bd","Type":"ContainerStarted","Data":"25dc43cef5cd00533294118d60d2a5a3478eff6adc5f190bda64c9d003ed02a6"} Jul 09 13:30:08.939445 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:08.939411 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:30:08.939687 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:08.939426 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:30:08.939687 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:08.939553 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:30:08.939687 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:08.939554 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65fnv" podUID="fb20e4df-4233-4fb6-a1f7-e58a789e3d56" Jul 09 13:30:08.939687 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:08.939651 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5zl2x" podUID="fbe44bc5-d23a-4535-b1d8-74ec52d0ce47" Jul 09 13:30:08.939923 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:08.939721 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6qqn" podUID="9e8320e4-2106-4edf-b225-d65a4b3eae55" Jul 09 13:30:09.095326 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:09.095287 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" event={"ID":"a74fe7a2-2170-47f0-aad8-6764d8ab41da","Type":"ContainerStarted","Data":"153be493b28107c74cc0094b22145ea7e0c9430f7e5c5b04e34102376b8c650b"} Jul 09 13:30:09.116772 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:09.116719 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xjchd" podStartSLOduration=4.864207472 podStartE2EDuration="26.116699769s" totalImagesPullingTime="21.252492297s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:29:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:09.116194559 +0000 UTC m=+26.621496853" watchObservedRunningTime="2026-07-09 13:30:09.116699769 +0000 UTC m=+26.622002037" Jul 09 13:30:10.940160 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:10.939952 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:30:10.940682 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:10.939985 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:30:10.940682 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:10.940249 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65fnv" podUID="fb20e4df-4233-4fb6-a1f7-e58a789e3d56" Jul 09 13:30:10.940682 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:10.939985 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:30:10.940682 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:10.940333 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6qqn" podUID="9e8320e4-2106-4edf-b225-d65a4b3eae55" Jul 09 13:30:10.940682 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:10.940440 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5zl2x" podUID="fbe44bc5-d23a-4535-b1d8-74ec52d0ce47" Jul 09 13:30:11.103579 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:11.103504 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmms8_3b512d09-0288-4fea-864e-fdc734afb1bd/ovn-acl-logging/0.log" Jul 09 13:30:11.103923 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:11.103889 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" event={"ID":"3b512d09-0288-4fea-864e-fdc734afb1bd","Type":"ContainerStarted","Data":"8c632f3ae34054d7be06deb7a7792231c90c0bebd5e9ac5ad794c9a7ffdb0644"} Jul 09 13:30:11.104205 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:11.104181 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:30:11.104354 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:11.104264 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:30:11.104427 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:11.104409 2573 scope.go:117] "RemoveContainer" containerID="c0faf4dad177e85fead341499327e7ef4cdbcfaf5abb7505722489bb82afda19" Jul 09 13:30:11.106228 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:11.106206 2573 generic.go:356] "Generic (PLEG): container finished" podID="8556c373-2549-4328-aa41-2a05076a088a" containerID="4769bc72ad095717795beb065f66fcc3b88cf81336fcb104f871eeb77e85a27c" exitCode=0 Jul 09 13:30:11.106325 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:11.106251 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v7rlv" event={"ID":"8556c373-2549-4328-aa41-2a05076a088a","Type":"ContainerDied","Data":"4769bc72ad095717795beb065f66fcc3b88cf81336fcb104f871eeb77e85a27c"} Jul 09 13:30:11.121199 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:11.121170 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:30:11.124514 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:11.124493 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:30:11.225500 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:11.225458 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:30:12.113720 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:12.113691 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmms8_3b512d09-0288-4fea-864e-fdc734afb1bd/ovn-acl-logging/0.log" Jul 09 13:30:12.114249 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:12.114036 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" event={"ID":"3b512d09-0288-4fea-864e-fdc734afb1bd","Type":"ContainerStarted","Data":"9fb0d24104a9849452d9a95fd0205121b4206a8040b4b9a9fa443d11bd160f23"} Jul 09 13:30:12.155007 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:12.154952 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" podStartSLOduration=9.480514677 podStartE2EDuration="29.154937472s" totalImagesPullingTime="19.674422795s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:29:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:12.154700257 +0000 UTC m=+29.660002544" watchObservedRunningTime="2026-07-09 13:30:12.154937472 +0000 UTC m=+29.660239739" Jul 09 13:30:12.940484 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:12.940451 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:30:12.940636 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:12.940551 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65fnv" podUID="fb20e4df-4233-4fb6-a1f7-e58a789e3d56" Jul 09 13:30:12.940636 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:12.940566 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:30:12.940636 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:12.940597 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:30:12.940761 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:12.940675 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5zl2x" podUID="fbe44bc5-d23a-4535-b1d8-74ec52d0ce47" Jul 09 13:30:12.940761 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:12.940746 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6qqn" podUID="9e8320e4-2106-4edf-b225-d65a4b3eae55" Jul 09 13:30:13.118543 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:13.118508 2573 generic.go:356] "Generic (PLEG): container finished" podID="8556c373-2549-4328-aa41-2a05076a088a" containerID="3f74221dea68a7a39759752c5cfc162eeb650e43f7997043195f83f729ba546f" exitCode=0 Jul 09 13:30:13.119005 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:13.118575 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v7rlv" event={"ID":"8556c373-2549-4328-aa41-2a05076a088a","Type":"ContainerDied","Data":"3f74221dea68a7a39759752c5cfc162eeb650e43f7997043195f83f729ba546f"} Jul 09 13:30:14.940591 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:14.940500 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:30:14.940591 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:14.940538 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:30:14.941065 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:14.940500 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:30:14.941065 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:14.940632 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6qqn" podUID="9e8320e4-2106-4edf-b225-d65a4b3eae55" Jul 09 13:30:14.941065 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:14.940687 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65fnv" podUID="fb20e4df-4233-4fb6-a1f7-e58a789e3d56" Jul 09 13:30:14.941065 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:14.940787 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5zl2x" podUID="fbe44bc5-d23a-4535-b1d8-74ec52d0ce47" Jul 09 13:30:15.125554 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:15.125510 2573 generic.go:356] "Generic (PLEG): container finished" podID="8556c373-2549-4328-aa41-2a05076a088a" containerID="0466f6130660ba66a01a3795dbfcf587ec77399c9616d41943faadf74befe819" exitCode=0 Jul 09 13:30:15.125709 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:15.125577 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v7rlv" event={"ID":"8556c373-2549-4328-aa41-2a05076a088a","Type":"ContainerDied","Data":"0466f6130660ba66a01a3795dbfcf587ec77399c9616d41943faadf74befe819"} Jul 09 13:30:16.682051 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:16.682001 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-metrics-certs\") pod \"network-metrics-daemon-5zl2x\" (UID: \"fbe44bc5-d23a-4535-b1d8-74ec52d0ce47\") " pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:30:16.682498 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:16.682076 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-original-pull-secret\") pod \"global-pull-secret-syncer-65fnv\" (UID: \"fb20e4df-4233-4fb6-a1f7-e58a789e3d56\") " pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:30:16.682498 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:16.682201 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 13:30:16.682498 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:16.682263 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-original-pull-secret podName:fb20e4df-4233-4fb6-a1f7-e58a789e3d56 nodeName:}" failed. No retries permitted until 2026-07-09 13:30:48.682247806 +0000 UTC m=+66.187550056 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-original-pull-secret") pod "global-pull-secret-syncer-65fnv" (UID: "fb20e4df-4233-4fb6-a1f7-e58a789e3d56") : object "kube-system"/"original-pull-secret" not registered Jul 09 13:30:16.682498 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:16.682201 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:30:16.682498 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:16.682380 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-metrics-certs podName:fbe44bc5-d23a-4535-b1d8-74ec52d0ce47 nodeName:}" failed. No retries permitted until 2026-07-09 13:30:48.682354223 +0000 UTC m=+66.187656486 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-metrics-certs") pod "network-metrics-daemon-5zl2x" (UID: "fbe44bc5-d23a-4535-b1d8-74ec52d0ce47") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:30:16.783036 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:16.782988 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjqpc\" (UniqueName: \"kubernetes.io/projected/9e8320e4-2106-4edf-b225-d65a4b3eae55-kube-api-access-wjqpc\") pod \"network-check-target-h6qqn\" (UID: \"9e8320e4-2106-4edf-b225-d65a4b3eae55\") " pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:30:16.783227 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:16.783165 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 13:30:16.783227 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:16.783195 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 13:30:16.783227 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:16.783209 2573 projected.go:196] Error preparing data for projected volume kube-api-access-wjqpc for pod openshift-network-diagnostics/network-check-target-h6qqn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:30:16.783333 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:16.783269 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e8320e4-2106-4edf-b225-d65a4b3eae55-kube-api-access-wjqpc podName:9e8320e4-2106-4edf-b225-d65a4b3eae55 nodeName:}" failed. No retries permitted until 2026-07-09 13:30:48.783250995 +0000 UTC m=+66.288553246 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wjqpc" (UniqueName: "kubernetes.io/projected/9e8320e4-2106-4edf-b225-d65a4b3eae55-kube-api-access-wjqpc") pod "network-check-target-h6qqn" (UID: "9e8320e4-2106-4edf-b225-d65a4b3eae55") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:30:16.939929 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:16.939846 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:30:16.940115 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:16.939846 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:30:16.940115 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:16.939846 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:30:16.940115 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:16.939991 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65fnv" podUID="fb20e4df-4233-4fb6-a1f7-e58a789e3d56" Jul 09 13:30:16.940115 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:16.940101 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5zl2x" podUID="fbe44bc5-d23a-4535-b1d8-74ec52d0ce47" Jul 09 13:30:16.940272 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:16.940192 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6qqn" podUID="9e8320e4-2106-4edf-b225-d65a4b3eae55" Jul 09 13:30:18.940418 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:18.940201 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:30:18.940863 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:18.940201 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:30:18.940863 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:18.940523 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65fnv" podUID="fb20e4df-4233-4fb6-a1f7-e58a789e3d56" Jul 09 13:30:18.940863 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:18.940273 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:30:18.940863 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:18.940572 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5zl2x" podUID="fbe44bc5-d23a-4535-b1d8-74ec52d0ce47" Jul 09 13:30:18.940863 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:18.940714 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6qqn" podUID="9e8320e4-2106-4edf-b225-d65a4b3eae55" Jul 09 13:30:20.139092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:20.139060 2573 generic.go:356] "Generic (PLEG): container finished" podID="8556c373-2549-4328-aa41-2a05076a088a" containerID="0271842b7a1cd1d9f3e1556a45d020145c55db602da614085e23fdff3873f7bf" exitCode=0 Jul 09 13:30:20.139526 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:20.139104 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v7rlv" event={"ID":"8556c373-2549-4328-aa41-2a05076a088a","Type":"ContainerDied","Data":"0271842b7a1cd1d9f3e1556a45d020145c55db602da614085e23fdff3873f7bf"} Jul 09 13:30:20.940435 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:20.940398 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:30:20.940435 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:20.940413 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:30:20.940663 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:20.940522 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65fnv" podUID="fb20e4df-4233-4fb6-a1f7-e58a789e3d56" Jul 09 13:30:20.940663 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:20.940579 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5zl2x" podUID="fbe44bc5-d23a-4535-b1d8-74ec52d0ce47" Jul 09 13:30:20.940663 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:20.940589 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:30:20.940806 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:20.940681 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6qqn" podUID="9e8320e4-2106-4edf-b225-d65a4b3eae55" Jul 09 13:30:21.144160 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:21.144126 2573 generic.go:356] "Generic (PLEG): container finished" podID="8556c373-2549-4328-aa41-2a05076a088a" containerID="d40a66e95b79304a08ca3d40f1c76a8b5c1fcf547ff27f4e9d853c81ecf33ce9" exitCode=0 Jul 09 13:30:21.144542 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:21.144177 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v7rlv" event={"ID":"8556c373-2549-4328-aa41-2a05076a088a","Type":"ContainerDied","Data":"d40a66e95b79304a08ca3d40f1c76a8b5c1fcf547ff27f4e9d853c81ecf33ce9"} Jul 09 13:30:22.148997 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:22.148962 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v7rlv" event={"ID":"8556c373-2549-4328-aa41-2a05076a088a","Type":"ContainerStarted","Data":"74148928be4f436322d48b5a84c5d9f99f5cfa306f41c6d88f63a55dabe66ea9"} Jul 09 13:30:22.176910 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:22.176859 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-v7rlv" podStartSLOduration=6.380806317 podStartE2EDuration="39.176841302s" totalImagesPullingTime="32.796034985s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:29:43 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:22.175736338 +0000 UTC m=+39.681038625" watchObservedRunningTime="2026-07-09 13:30:22.176841302 +0000 UTC m=+39.682143567" Jul 09 13:30:22.709938 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:22.709905 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-65fnv"] Jul 09 13:30:22.710144 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:22.710076 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:30:22.710218 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:22.710193 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65fnv" podUID="fb20e4df-4233-4fb6-a1f7-e58a789e3d56" Jul 09 13:30:22.710918 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:22.710886 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-h6qqn"] Jul 09 13:30:22.711070 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:22.710987 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:30:22.711137 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:22.711087 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6qqn" podUID="9e8320e4-2106-4edf-b225-d65a4b3eae55" Jul 09 13:30:22.711676 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:22.711651 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5zl2x"] Jul 09 13:30:22.711763 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:22.711748 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:30:22.711846 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:22.711829 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5zl2x" podUID="fbe44bc5-d23a-4535-b1d8-74ec52d0ce47" Jul 09 13:30:23.939972 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:23.939934 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:30:23.940649 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:23.940071 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5zl2x" podUID="fbe44bc5-d23a-4535-b1d8-74ec52d0ce47" Jul 09 13:30:24.939754 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:24.939713 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:30:24.939925 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:24.939762 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:30:24.939925 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:24.939854 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65fnv" podUID="fb20e4df-4233-4fb6-a1f7-e58a789e3d56" Jul 09 13:30:24.940275 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:24.939974 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6qqn" podUID="9e8320e4-2106-4edf-b225-d65a4b3eae55" Jul 09 13:30:25.940179 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:25.940139 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:30:25.940628 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:25.940281 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5zl2x" podUID="fbe44bc5-d23a-4535-b1d8-74ec52d0ce47" Jul 09 13:30:26.940100 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:26.940060 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:30:26.940100 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:26.940095 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:30:26.940555 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:26.940189 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h6qqn" podUID="9e8320e4-2106-4edf-b225-d65a4b3eae55" Jul 09 13:30:26.940555 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:26.940306 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-65fnv" podUID="fb20e4df-4233-4fb6-a1f7-e58a789e3d56" Jul 09 13:30:27.325093 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.324807 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-87.ec2.internal" event="NodeReady" Jul 09 13:30:27.325235 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.325148 2573 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 13:30:27.395513 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.395481 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9bhjn"] Jul 09 13:30:27.416344 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.416310 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tllnp"] Jul 09 13:30:27.416508 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.416437 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9bhjn" Jul 09 13:30:27.420663 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.420638 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 13:30:27.420812 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.420710 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vw8bp\"" Jul 09 13:30:27.420873 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.420809 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 13:30:27.448291 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.448254 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9bhjn"] Jul 09 13:30:27.448291 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.448304 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tllnp"] Jul 09 13:30:27.448554 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.448434 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tllnp" Jul 09 13:30:27.451339 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.451308 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 13:30:27.451468 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.451425 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 13:30:27.451701 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.451686 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 13:30:27.452672 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.452652 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 13:30:27.452800 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.452782 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kc8dw\"" Jul 09 13:30:27.504120 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.504086 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qp49h"] Jul 09 13:30:27.522219 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.522192 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qp49h"] Jul 09 13:30:27.522380 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.522293 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qp49h" Jul 09 13:30:27.526166 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.526144 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 13:30:27.526166 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.526159 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 13:30:27.526670 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.526652 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 13:30:27.526730 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.526697 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-6tscd\"" Jul 09 13:30:27.562090 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.562044 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56jdx\" (UniqueName: \"kubernetes.io/projected/17a28000-664b-4eba-bd87-5c67029f4358-kube-api-access-56jdx\") pod \"insights-runtime-extractor-tllnp\" (UID: \"17a28000-664b-4eba-bd87-5c67029f4358\") " pod="openshift-insights/insights-runtime-extractor-tllnp" Jul 09 13:30:27.562260 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.562105 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/17a28000-664b-4eba-bd87-5c67029f4358-data-volume\") pod \"insights-runtime-extractor-tllnp\" (UID: \"17a28000-664b-4eba-bd87-5c67029f4358\") " pod="openshift-insights/insights-runtime-extractor-tllnp" Jul 09 13:30:27.562260 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.562151 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/17a28000-664b-4eba-bd87-5c67029f4358-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tllnp\" (UID: \"17a28000-664b-4eba-bd87-5c67029f4358\") " pod="openshift-insights/insights-runtime-extractor-tllnp" Jul 09 13:30:27.562260 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.562215 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/07c548d3-07f4-41f8-b82c-22932794c741-config-volume\") pod \"dns-default-9bhjn\" (UID: \"07c548d3-07f4-41f8-b82c-22932794c741\") " pod="openshift-dns/dns-default-9bhjn" Jul 09 13:30:27.562260 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.562234 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07c548d3-07f4-41f8-b82c-22932794c741-tmp-dir\") pod \"dns-default-9bhjn\" (UID: \"07c548d3-07f4-41f8-b82c-22932794c741\") " pod="openshift-dns/dns-default-9bhjn" Jul 09 13:30:27.562260 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.562255 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfn8b\" (UniqueName: \"kubernetes.io/projected/07c548d3-07f4-41f8-b82c-22932794c741-kube-api-access-hfn8b\") pod \"dns-default-9bhjn\" (UID: \"07c548d3-07f4-41f8-b82c-22932794c741\") " pod="openshift-dns/dns-default-9bhjn" Jul 09 13:30:27.562419 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.562313 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/07c548d3-07f4-41f8-b82c-22932794c741-metrics-tls\") pod \"dns-default-9bhjn\" (UID: \"07c548d3-07f4-41f8-b82c-22932794c741\") " pod="openshift-dns/dns-default-9bhjn" Jul 09 13:30:27.562419 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.562338 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/17a28000-664b-4eba-bd87-5c67029f4358-crio-socket\") pod \"insights-runtime-extractor-tllnp\" (UID: \"17a28000-664b-4eba-bd87-5c67029f4358\") " pod="openshift-insights/insights-runtime-extractor-tllnp" Jul 09 13:30:27.562419 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.562391 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/17a28000-664b-4eba-bd87-5c67029f4358-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tllnp\" (UID: \"17a28000-664b-4eba-bd87-5c67029f4358\") " pod="openshift-insights/insights-runtime-extractor-tllnp" Jul 09 13:30:27.663366 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.663333 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/07c548d3-07f4-41f8-b82c-22932794c741-metrics-tls\") pod \"dns-default-9bhjn\" (UID: \"07c548d3-07f4-41f8-b82c-22932794c741\") " pod="openshift-dns/dns-default-9bhjn" Jul 09 13:30:27.663366 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.663371 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/17a28000-664b-4eba-bd87-5c67029f4358-crio-socket\") pod \"insights-runtime-extractor-tllnp\" (UID: \"17a28000-664b-4eba-bd87-5c67029f4358\") " pod="openshift-insights/insights-runtime-extractor-tllnp" Jul 09 13:30:27.663601 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.663407 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/17a28000-664b-4eba-bd87-5c67029f4358-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tllnp\" (UID: \"17a28000-664b-4eba-bd87-5c67029f4358\") " pod="openshift-insights/insights-runtime-extractor-tllnp" Jul 09 13:30:27.663601 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.663437 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56jdx\" (UniqueName: \"kubernetes.io/projected/17a28000-664b-4eba-bd87-5c67029f4358-kube-api-access-56jdx\") pod \"insights-runtime-extractor-tllnp\" (UID: \"17a28000-664b-4eba-bd87-5c67029f4358\") " pod="openshift-insights/insights-runtime-extractor-tllnp" Jul 09 13:30:27.663601 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.663458 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rswnf\" (UniqueName: \"kubernetes.io/projected/86d778c7-615a-4398-b5ff-5d4ca5cfa2e2-kube-api-access-rswnf\") pod \"ingress-canary-qp49h\" (UID: \"86d778c7-615a-4398-b5ff-5d4ca5cfa2e2\") " pod="openshift-ingress-canary/ingress-canary-qp49h" Jul 09 13:30:27.663601 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.663478 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86d778c7-615a-4398-b5ff-5d4ca5cfa2e2-cert\") pod \"ingress-canary-qp49h\" (UID: \"86d778c7-615a-4398-b5ff-5d4ca5cfa2e2\") " pod="openshift-ingress-canary/ingress-canary-qp49h" Jul 09 13:30:27.663601 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.663518 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/17a28000-664b-4eba-bd87-5c67029f4358-data-volume\") pod \"insights-runtime-extractor-tllnp\" (UID: \"17a28000-664b-4eba-bd87-5c67029f4358\") " pod="openshift-insights/insights-runtime-extractor-tllnp" Jul 09 13:30:27.663601 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.663539 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/17a28000-664b-4eba-bd87-5c67029f4358-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tllnp\" (UID: \"17a28000-664b-4eba-bd87-5c67029f4358\") " pod="openshift-insights/insights-runtime-extractor-tllnp" Jul 09 13:30:27.663601 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.663564 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/07c548d3-07f4-41f8-b82c-22932794c741-config-volume\") pod \"dns-default-9bhjn\" (UID: \"07c548d3-07f4-41f8-b82c-22932794c741\") " pod="openshift-dns/dns-default-9bhjn" Jul 09 13:30:27.663601 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.663583 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07c548d3-07f4-41f8-b82c-22932794c741-tmp-dir\") pod \"dns-default-9bhjn\" (UID: \"07c548d3-07f4-41f8-b82c-22932794c741\") " pod="openshift-dns/dns-default-9bhjn" Jul 09 13:30:27.663934 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.663608 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfn8b\" (UniqueName: \"kubernetes.io/projected/07c548d3-07f4-41f8-b82c-22932794c741-kube-api-access-hfn8b\") pod \"dns-default-9bhjn\" (UID: \"07c548d3-07f4-41f8-b82c-22932794c741\") " pod="openshift-dns/dns-default-9bhjn" Jul 09 13:30:27.663934 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.663671 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/17a28000-664b-4eba-bd87-5c67029f4358-crio-socket\") pod \"insights-runtime-extractor-tllnp\" (UID: \"17a28000-664b-4eba-bd87-5c67029f4358\") " pod="openshift-insights/insights-runtime-extractor-tllnp" Jul 09 13:30:27.664000 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.663976 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/17a28000-664b-4eba-bd87-5c67029f4358-data-volume\") pod \"insights-runtime-extractor-tllnp\" (UID: \"17a28000-664b-4eba-bd87-5c67029f4358\") " pod="openshift-insights/insights-runtime-extractor-tllnp" Jul 09 13:30:27.664188 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.664162 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07c548d3-07f4-41f8-b82c-22932794c741-tmp-dir\") pod \"dns-default-9bhjn\" (UID: \"07c548d3-07f4-41f8-b82c-22932794c741\") " pod="openshift-dns/dns-default-9bhjn" Jul 09 13:30:27.664267 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.664216 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/07c548d3-07f4-41f8-b82c-22932794c741-config-volume\") pod \"dns-default-9bhjn\" (UID: \"07c548d3-07f4-41f8-b82c-22932794c741\") " pod="openshift-dns/dns-default-9bhjn" Jul 09 13:30:27.664327 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.664309 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/17a28000-664b-4eba-bd87-5c67029f4358-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tllnp\" (UID: \"17a28000-664b-4eba-bd87-5c67029f4358\") " pod="openshift-insights/insights-runtime-extractor-tllnp" Jul 09 13:30:27.667666 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.667645 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/17a28000-664b-4eba-bd87-5c67029f4358-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tllnp\" (UID: \"17a28000-664b-4eba-bd87-5c67029f4358\") " pod="openshift-insights/insights-runtime-extractor-tllnp" Jul 09 13:30:27.667762 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.667656 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/07c548d3-07f4-41f8-b82c-22932794c741-metrics-tls\") pod \"dns-default-9bhjn\" (UID: \"07c548d3-07f4-41f8-b82c-22932794c741\") " pod="openshift-dns/dns-default-9bhjn" Jul 09 13:30:27.676816 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.676791 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfn8b\" (UniqueName: \"kubernetes.io/projected/07c548d3-07f4-41f8-b82c-22932794c741-kube-api-access-hfn8b\") pod \"dns-default-9bhjn\" (UID: \"07c548d3-07f4-41f8-b82c-22932794c741\") " pod="openshift-dns/dns-default-9bhjn" Jul 09 13:30:27.678413 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.678395 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56jdx\" (UniqueName: \"kubernetes.io/projected/17a28000-664b-4eba-bd87-5c67029f4358-kube-api-access-56jdx\") pod \"insights-runtime-extractor-tllnp\" (UID: \"17a28000-664b-4eba-bd87-5c67029f4358\") " pod="openshift-insights/insights-runtime-extractor-tllnp" Jul 09 13:30:27.728059 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.728001 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9bhjn" Jul 09 13:30:27.757775 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.757744 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tllnp" Jul 09 13:30:27.764101 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.764073 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86d778c7-615a-4398-b5ff-5d4ca5cfa2e2-cert\") pod \"ingress-canary-qp49h\" (UID: \"86d778c7-615a-4398-b5ff-5d4ca5cfa2e2\") " pod="openshift-ingress-canary/ingress-canary-qp49h" Jul 09 13:30:27.764248 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.764156 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rswnf\" (UniqueName: \"kubernetes.io/projected/86d778c7-615a-4398-b5ff-5d4ca5cfa2e2-kube-api-access-rswnf\") pod \"ingress-canary-qp49h\" (UID: \"86d778c7-615a-4398-b5ff-5d4ca5cfa2e2\") " pod="openshift-ingress-canary/ingress-canary-qp49h" Jul 09 13:30:27.766539 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.766510 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86d778c7-615a-4398-b5ff-5d4ca5cfa2e2-cert\") pod \"ingress-canary-qp49h\" (UID: \"86d778c7-615a-4398-b5ff-5d4ca5cfa2e2\") " pod="openshift-ingress-canary/ingress-canary-qp49h" Jul 09 13:30:27.774837 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.774808 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rswnf\" (UniqueName: \"kubernetes.io/projected/86d778c7-615a-4398-b5ff-5d4ca5cfa2e2-kube-api-access-rswnf\") pod \"ingress-canary-qp49h\" (UID: \"86d778c7-615a-4398-b5ff-5d4ca5cfa2e2\") " pod="openshift-ingress-canary/ingress-canary-qp49h" Jul 09 13:30:27.831830 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.831593 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qp49h" Jul 09 13:30:27.919129 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.919057 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9bhjn"] Jul 09 13:30:27.921922 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.921886 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tllnp"] Jul 09 13:30:27.923567 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:30:27.923529 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07c548d3_07f4_41f8_b82c_22932794c741.slice/crio-0bcbd641a4d5d7eb99b7f1e76ed7e955b5886bfbd49eaee7741150953a4a0634 WatchSource:0}: Error finding container 0bcbd641a4d5d7eb99b7f1e76ed7e955b5886bfbd49eaee7741150953a4a0634: Status 404 returned error can't find the container with id 0bcbd641a4d5d7eb99b7f1e76ed7e955b5886bfbd49eaee7741150953a4a0634 Jul 09 13:30:27.925359 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:30:27.925337 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17a28000_664b_4eba_bd87_5c67029f4358.slice/crio-619fba1ebfa92b9b72fba71691cff97a7438559ec56420b48443659d86d578b4 WatchSource:0}: Error finding container 619fba1ebfa92b9b72fba71691cff97a7438559ec56420b48443659d86d578b4: Status 404 returned error can't find the container with id 619fba1ebfa92b9b72fba71691cff97a7438559ec56420b48443659d86d578b4 Jul 09 13:30:27.940377 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.940337 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:30:27.943550 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.943528 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 13:30:27.943661 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.943559 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wmhp6\"" Jul 09 13:30:27.990844 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:27.990818 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qp49h"] Jul 09 13:30:27.992717 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:30:27.992692 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86d778c7_615a_4398_b5ff_5d4ca5cfa2e2.slice/crio-4da1a691ff99388c290c98f90c1a94bc74f29fc80b15cdc3ab8f0c6c34b23e04 WatchSource:0}: Error finding container 4da1a691ff99388c290c98f90c1a94bc74f29fc80b15cdc3ab8f0c6c34b23e04: Status 404 returned error can't find the container with id 4da1a691ff99388c290c98f90c1a94bc74f29fc80b15cdc3ab8f0c6c34b23e04 Jul 09 13:30:28.163741 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:28.163700 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qp49h" event={"ID":"86d778c7-615a-4398-b5ff-5d4ca5cfa2e2","Type":"ContainerStarted","Data":"4da1a691ff99388c290c98f90c1a94bc74f29fc80b15cdc3ab8f0c6c34b23e04"} Jul 09 13:30:28.164705 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:28.164676 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tllnp" event={"ID":"17a28000-664b-4eba-bd87-5c67029f4358","Type":"ContainerStarted","Data":"619fba1ebfa92b9b72fba71691cff97a7438559ec56420b48443659d86d578b4"} Jul 09 13:30:28.165579 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:28.165556 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9bhjn" event={"ID":"07c548d3-07f4-41f8-b82c-22932794c741","Type":"ContainerStarted","Data":"0bcbd641a4d5d7eb99b7f1e76ed7e955b5886bfbd49eaee7741150953a4a0634"} Jul 09 13:30:28.944813 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:28.944262 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:30:28.944813 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:28.944575 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:30:28.947596 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:28.947574 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 13:30:28.948566 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:28.948544 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 13:30:28.948805 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:28.948788 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 13:30:28.949051 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:28.949034 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xg9bl\"" Jul 09 13:30:32.237786 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.237506 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6578f4b68b-pwb9x"] Jul 09 13:30:32.252189 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.252160 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:30:32.256370 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.256344 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 09 13:30:32.257636 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.257404 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6578f4b68b-pwb9x"] Jul 09 13:30:32.257636 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.257459 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 09 13:30:32.257636 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.257524 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 09 13:30:32.257636 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.257527 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 09 13:30:32.257636 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.257527 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 09 13:30:32.257636 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.257597 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-z6pxn\"" Jul 09 13:30:32.258029 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.257901 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 09 13:30:32.258029 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.257907 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 09 13:30:32.401564 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.401526 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f428915b-73cc-4e64-8d11-cdd4c4ab2038-oauth-serving-cert\") pod \"console-6578f4b68b-pwb9x\" (UID: \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\") " pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:30:32.401758 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.401592 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f428915b-73cc-4e64-8d11-cdd4c4ab2038-console-config\") pod \"console-6578f4b68b-pwb9x\" (UID: \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\") " pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:30:32.401758 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.401622 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f428915b-73cc-4e64-8d11-cdd4c4ab2038-service-ca\") pod \"console-6578f4b68b-pwb9x\" (UID: \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\") " pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:30:32.401758 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.401645 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f428915b-73cc-4e64-8d11-cdd4c4ab2038-console-oauth-config\") pod \"console-6578f4b68b-pwb9x\" (UID: \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\") " pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:30:32.401758 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.401667 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzs54\" (UniqueName: \"kubernetes.io/projected/f428915b-73cc-4e64-8d11-cdd4c4ab2038-kube-api-access-mzs54\") pod \"console-6578f4b68b-pwb9x\" (UID: \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\") " pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:30:32.401758 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.401697 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f428915b-73cc-4e64-8d11-cdd4c4ab2038-console-serving-cert\") pod \"console-6578f4b68b-pwb9x\" (UID: \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\") " pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:30:32.454443 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.454409 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-p5lxr"] Jul 09 13:30:32.478210 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.478179 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-p5lxr"] Jul 09 13:30:32.478365 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.478312 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-p5lxr" Jul 09 13:30:32.481086 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.481063 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 09 13:30:32.481233 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.481178 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-8xbg8\"" Jul 09 13:30:32.502850 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.502774 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f428915b-73cc-4e64-8d11-cdd4c4ab2038-oauth-serving-cert\") pod \"console-6578f4b68b-pwb9x\" (UID: \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\") " pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:30:32.503037 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.502855 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f428915b-73cc-4e64-8d11-cdd4c4ab2038-console-config\") pod \"console-6578f4b68b-pwb9x\" (UID: \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\") " pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:30:32.503037 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.502881 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f428915b-73cc-4e64-8d11-cdd4c4ab2038-service-ca\") pod \"console-6578f4b68b-pwb9x\" (UID: \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\") " pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:30:32.503037 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.502908 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f428915b-73cc-4e64-8d11-cdd4c4ab2038-console-oauth-config\") pod \"console-6578f4b68b-pwb9x\" (UID: \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\") " pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:30:32.503037 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.502932 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzs54\" (UniqueName: \"kubernetes.io/projected/f428915b-73cc-4e64-8d11-cdd4c4ab2038-kube-api-access-mzs54\") pod \"console-6578f4b68b-pwb9x\" (UID: \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\") " pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:30:32.503037 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.502963 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f428915b-73cc-4e64-8d11-cdd4c4ab2038-console-serving-cert\") pod \"console-6578f4b68b-pwb9x\" (UID: \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\") " pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:30:32.503794 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.503768 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f428915b-73cc-4e64-8d11-cdd4c4ab2038-console-config\") pod \"console-6578f4b68b-pwb9x\" (UID: \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\") " pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:30:32.503913 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.503773 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f428915b-73cc-4e64-8d11-cdd4c4ab2038-oauth-serving-cert\") pod \"console-6578f4b68b-pwb9x\" (UID: \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\") " pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:30:32.503913 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.503822 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f428915b-73cc-4e64-8d11-cdd4c4ab2038-service-ca\") pod \"console-6578f4b68b-pwb9x\" (UID: \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\") " pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:30:32.506673 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.506645 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f428915b-73cc-4e64-8d11-cdd4c4ab2038-console-oauth-config\") pod \"console-6578f4b68b-pwb9x\" (UID: \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\") " pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:30:32.506785 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.506729 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f428915b-73cc-4e64-8d11-cdd4c4ab2038-console-serving-cert\") pod \"console-6578f4b68b-pwb9x\" (UID: \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\") " pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:30:32.513705 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.513668 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzs54\" (UniqueName: \"kubernetes.io/projected/f428915b-73cc-4e64-8d11-cdd4c4ab2038-kube-api-access-mzs54\") pod \"console-6578f4b68b-pwb9x\" (UID: \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\") " pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:30:32.564498 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.564452 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:30:32.604297 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.604271 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b7469638-03a3-4bd5-a128-a36e3bd5353e-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-p5lxr\" (UID: \"b7469638-03a3-4bd5-a128-a36e3bd5353e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-p5lxr" Jul 09 13:30:32.705280 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:32.705244 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b7469638-03a3-4bd5-a128-a36e3bd5353e-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-p5lxr\" (UID: \"b7469638-03a3-4bd5-a128-a36e3bd5353e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-p5lxr" Jul 09 13:30:32.705493 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:32.705398 2573 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 09 13:30:32.705493 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:32.705482 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7469638-03a3-4bd5-a128-a36e3bd5353e-tls-certificates podName:b7469638-03a3-4bd5-a128-a36e3bd5353e nodeName:}" failed. No retries permitted until 2026-07-09 13:30:33.205466286 +0000 UTC m=+50.710768532 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/b7469638-03a3-4bd5-a128-a36e3bd5353e-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-p5lxr" (UID: "b7469638-03a3-4bd5-a128-a36e3bd5353e") : secret "prometheus-operator-admission-webhook-tls" not found Jul 09 13:30:33.209579 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:33.209541 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b7469638-03a3-4bd5-a128-a36e3bd5353e-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-p5lxr\" (UID: \"b7469638-03a3-4bd5-a128-a36e3bd5353e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-p5lxr" Jul 09 13:30:33.212556 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:33.212522 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b7469638-03a3-4bd5-a128-a36e3bd5353e-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-p5lxr\" (UID: \"b7469638-03a3-4bd5-a128-a36e3bd5353e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-p5lxr" Jul 09 13:30:33.389427 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:33.389383 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-p5lxr" Jul 09 13:30:34.325154 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:34.324908 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-p5lxr"] Jul 09 13:30:34.328352 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:34.328277 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6578f4b68b-pwb9x"] Jul 09 13:30:34.449604 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:30:34.449475 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7469638_03a3_4bd5_a128_a36e3bd5353e.slice/crio-9d196dd6f6318138909c9135624c743a3c2ea4659f9853735673b211d7407590 WatchSource:0}: Error finding container 9d196dd6f6318138909c9135624c743a3c2ea4659f9853735673b211d7407590: Status 404 returned error can't find the container with id 9d196dd6f6318138909c9135624c743a3c2ea4659f9853735673b211d7407590 Jul 09 13:30:34.450306 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:30:34.450275 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf428915b_73cc_4e64_8d11_cdd4c4ab2038.slice/crio-a7aa8f4a942cfb1a0d8d0ab43effce0f5412e2c786c81d371cc829a0cf5ccd5c WatchSource:0}: Error finding container a7aa8f4a942cfb1a0d8d0ab43effce0f5412e2c786c81d371cc829a0cf5ccd5c: Status 404 returned error can't find the container with id a7aa8f4a942cfb1a0d8d0ab43effce0f5412e2c786c81d371cc829a0cf5ccd5c Jul 09 13:30:35.189779 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:35.189484 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9bhjn" event={"ID":"07c548d3-07f4-41f8-b82c-22932794c741","Type":"ContainerStarted","Data":"e92fe20d002954c92d2039cf63d44136e0cda6043d55f0853d49aadcbe1dc2c5"} Jul 09 13:30:35.189779 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:35.189560 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9bhjn" event={"ID":"07c548d3-07f4-41f8-b82c-22932794c741","Type":"ContainerStarted","Data":"82e73d08765ffa28aecac0776010899f70fb32dca374a81b1020413792a2ca33"} Jul 09 13:30:35.189779 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:35.189592 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9bhjn" Jul 09 13:30:35.192511 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:35.191940 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qp49h" event={"ID":"86d778c7-615a-4398-b5ff-5d4ca5cfa2e2","Type":"ContainerStarted","Data":"ff79d24cab77548ef4c2a097de2337b0365287249ba9a42829b10960566c6cd4"} Jul 09 13:30:35.194140 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:35.194050 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tllnp" event={"ID":"17a28000-664b-4eba-bd87-5c67029f4358","Type":"ContainerStarted","Data":"f73fdd5c7e63558639d1efa9c35d267ffaa0eb648f30b440d0e53c94f6cb4691"} Jul 09 13:30:35.195806 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:35.195758 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-p5lxr" event={"ID":"b7469638-03a3-4bd5-a128-a36e3bd5353e","Type":"ContainerStarted","Data":"9d196dd6f6318138909c9135624c743a3c2ea4659f9853735673b211d7407590"} Jul 09 13:30:35.197256 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:35.197225 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6578f4b68b-pwb9x" event={"ID":"f428915b-73cc-4e64-8d11-cdd4c4ab2038","Type":"ContainerStarted","Data":"a7aa8f4a942cfb1a0d8d0ab43effce0f5412e2c786c81d371cc829a0cf5ccd5c"} Jul 09 13:30:35.211996 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:35.211231 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9bhjn" podStartSLOduration=1.968549185 podStartE2EDuration="8.211209833s" totalImagesPullingTime="6.242660648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:30:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:35.209914948 +0000 UTC m=+52.715217243" watchObservedRunningTime="2026-07-09 13:30:35.211209833 +0000 UTC m=+52.716512101" Jul 09 13:30:35.228828 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:35.228771 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qp49h" podStartSLOduration=2.052879216 podStartE2EDuration="8.228756038s" totalImagesPullingTime="6.175876822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:30:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:35.227645378 +0000 UTC m=+52.732947649" watchObservedRunningTime="2026-07-09 13:30:35.228756038 +0000 UTC m=+52.734058309" Jul 09 13:30:36.203312 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:36.203271 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tllnp" event={"ID":"17a28000-664b-4eba-bd87-5c67029f4358","Type":"ContainerStarted","Data":"6ff39122fc4de1f146f913c2b931af75ce0b116e2c9cbdc42b91b05e406e2e43"} Jul 09 13:30:37.208004 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:37.207968 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-p5lxr" event={"ID":"b7469638-03a3-4bd5-a128-a36e3bd5353e","Type":"ContainerStarted","Data":"7f247cceb25d79385186ddaf70eca3862af6937ae5b3b52833142daf07630421"} Jul 09 13:30:37.208441 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:37.208259 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-p5lxr" Jul 09 13:30:37.213717 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:37.213688 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-p5lxr" Jul 09 13:30:37.230229 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:37.230173 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-p5lxr" podStartSLOduration=3.405205076 podStartE2EDuration="5.230156087s" totalImagesPullingTime="1.824951011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:30:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:37.228438447 +0000 UTC m=+54.733740715" watchObservedRunningTime="2026-07-09 13:30:37.230156087 +0000 UTC m=+54.735458359" Jul 09 13:30:37.550368 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:37.550296 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-lwvt4"] Jul 09 13:30:37.563887 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:37.563859 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lwvt4" Jul 09 13:30:37.568322 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:37.568285 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 09 13:30:37.568482 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:37.568399 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 09 13:30:37.568733 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:37.568716 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 13:30:37.568932 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:37.568918 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 13:30:37.569262 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:37.569241 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-lwvt4"] Jul 09 13:30:37.569571 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:37.569554 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-rb2ls\"" Jul 09 13:30:37.569616 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:37.569560 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 13:30:37.744043 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:37.743989 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b125e19d-1900-4c3d-945a-61de5502b779-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-lwvt4\" (UID: \"b125e19d-1900-4c3d-945a-61de5502b779\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lwvt4" Jul 09 13:30:37.744214 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:37.744090 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b125e19d-1900-4c3d-945a-61de5502b779-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-lwvt4\" (UID: \"b125e19d-1900-4c3d-945a-61de5502b779\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lwvt4" Jul 09 13:30:37.744214 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:37.744134 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b125e19d-1900-4c3d-945a-61de5502b779-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-lwvt4\" (UID: \"b125e19d-1900-4c3d-945a-61de5502b779\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lwvt4" Jul 09 13:30:37.744214 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:37.744161 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcmjv\" (UniqueName: \"kubernetes.io/projected/b125e19d-1900-4c3d-945a-61de5502b779-kube-api-access-pcmjv\") pod \"prometheus-operator-567f7b8b7c-lwvt4\" (UID: \"b125e19d-1900-4c3d-945a-61de5502b779\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lwvt4" Jul 09 13:30:37.845468 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:37.845368 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b125e19d-1900-4c3d-945a-61de5502b779-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-lwvt4\" (UID: \"b125e19d-1900-4c3d-945a-61de5502b779\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lwvt4" Jul 09 13:30:37.845468 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:37.845435 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b125e19d-1900-4c3d-945a-61de5502b779-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-lwvt4\" (UID: \"b125e19d-1900-4c3d-945a-61de5502b779\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lwvt4" Jul 09 13:30:37.845689 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:37.845628 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b125e19d-1900-4c3d-945a-61de5502b779-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-lwvt4\" (UID: \"b125e19d-1900-4c3d-945a-61de5502b779\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lwvt4" Jul 09 13:30:37.845689 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:37.845683 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcmjv\" (UniqueName: \"kubernetes.io/projected/b125e19d-1900-4c3d-945a-61de5502b779-kube-api-access-pcmjv\") pod \"prometheus-operator-567f7b8b7c-lwvt4\" (UID: \"b125e19d-1900-4c3d-945a-61de5502b779\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lwvt4" Jul 09 13:30:37.845801 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:37.845787 2573 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 09 13:30:37.845882 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:37.845862 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b125e19d-1900-4c3d-945a-61de5502b779-prometheus-operator-tls podName:b125e19d-1900-4c3d-945a-61de5502b779 nodeName:}" failed. No retries permitted until 2026-07-09 13:30:38.34584473 +0000 UTC m=+55.851146980 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/b125e19d-1900-4c3d-945a-61de5502b779-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-lwvt4" (UID: "b125e19d-1900-4c3d-945a-61de5502b779") : secret "prometheus-operator-tls" not found Jul 09 13:30:37.846364 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:37.846339 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b125e19d-1900-4c3d-945a-61de5502b779-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-lwvt4\" (UID: \"b125e19d-1900-4c3d-945a-61de5502b779\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lwvt4" Jul 09 13:30:37.850434 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:37.850288 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b125e19d-1900-4c3d-945a-61de5502b779-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-lwvt4\" (UID: \"b125e19d-1900-4c3d-945a-61de5502b779\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lwvt4" Jul 09 13:30:37.860450 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:37.860399 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcmjv\" (UniqueName: \"kubernetes.io/projected/b125e19d-1900-4c3d-945a-61de5502b779-kube-api-access-pcmjv\") pod \"prometheus-operator-567f7b8b7c-lwvt4\" (UID: \"b125e19d-1900-4c3d-945a-61de5502b779\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lwvt4" Jul 09 13:30:38.214361 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:38.214322 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6578f4b68b-pwb9x" event={"ID":"f428915b-73cc-4e64-8d11-cdd4c4ab2038","Type":"ContainerStarted","Data":"b89ab67ad58c0bf8943940b07efa711695f049c1964567aa4c2850412ce5f03c"} Jul 09 13:30:38.238065 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:38.237990 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6578f4b68b-pwb9x" podStartSLOduration=2.8251314560000003 podStartE2EDuration="6.237966455s" totalImagesPullingTime="3.412834999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:30:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:38.237463861 +0000 UTC m=+55.742766155" watchObservedRunningTime="2026-07-09 13:30:38.237966455 +0000 UTC m=+55.743268724" Jul 09 13:30:38.350521 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:38.350463 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b125e19d-1900-4c3d-945a-61de5502b779-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-lwvt4\" (UID: \"b125e19d-1900-4c3d-945a-61de5502b779\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lwvt4" Jul 09 13:30:38.353612 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:38.353565 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b125e19d-1900-4c3d-945a-61de5502b779-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-lwvt4\" (UID: \"b125e19d-1900-4c3d-945a-61de5502b779\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lwvt4" Jul 09 13:30:38.475167 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:38.475085 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lwvt4" Jul 09 13:30:38.629423 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:38.629386 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-lwvt4"] Jul 09 13:30:38.633212 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:30:38.633180 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb125e19d_1900_4c3d_945a_61de5502b779.slice/crio-f960df9d3091322d951912384fba20b8eeaaea9353fc0b8a4fa2a8333e1d2f78 WatchSource:0}: Error finding container f960df9d3091322d951912384fba20b8eeaaea9353fc0b8a4fa2a8333e1d2f78: Status 404 returned error can't find the container with id f960df9d3091322d951912384fba20b8eeaaea9353fc0b8a4fa2a8333e1d2f78 Jul 09 13:30:39.219011 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:39.218970 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lwvt4" event={"ID":"b125e19d-1900-4c3d-945a-61de5502b779","Type":"ContainerStarted","Data":"f960df9d3091322d951912384fba20b8eeaaea9353fc0b8a4fa2a8333e1d2f78"} Jul 09 13:30:40.204639 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.204602 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56c6f87bc5-824bh"] Jul 09 13:30:40.206822 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.206799 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:30:40.216791 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.216765 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 09 13:30:40.219433 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.219409 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56c6f87bc5-824bh"] Jul 09 13:30:40.225157 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.225130 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tllnp" event={"ID":"17a28000-664b-4eba-bd87-5c67029f4358","Type":"ContainerStarted","Data":"bdcdac7578ea06c8069c9899ba1018f5a73c2e95c7b2de0176046353349d682a"} Jul 09 13:30:40.264463 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.264402 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tllnp" podStartSLOduration=1.566386761 podStartE2EDuration="13.26438603s" totalImagesPullingTime="11.697999269s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:30:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:40.2627646 +0000 UTC m=+57.768066881" watchObservedRunningTime="2026-07-09 13:30:40.26438603 +0000 UTC m=+57.769688308" Jul 09 13:30:40.368595 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.368551 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff1595e1-86db-4f53-a118-dff9aa8c51cc-console-serving-cert\") pod \"console-56c6f87bc5-824bh\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:30:40.368771 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.368618 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9jkg\" (UniqueName: \"kubernetes.io/projected/ff1595e1-86db-4f53-a118-dff9aa8c51cc-kube-api-access-z9jkg\") pod \"console-56c6f87bc5-824bh\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:30:40.368771 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.368670 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff1595e1-86db-4f53-a118-dff9aa8c51cc-trusted-ca-bundle\") pod \"console-56c6f87bc5-824bh\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:30:40.368771 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.368702 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff1595e1-86db-4f53-a118-dff9aa8c51cc-console-oauth-config\") pod \"console-56c6f87bc5-824bh\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:30:40.368936 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.368766 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff1595e1-86db-4f53-a118-dff9aa8c51cc-console-config\") pod \"console-56c6f87bc5-824bh\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:30:40.368936 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.368888 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff1595e1-86db-4f53-a118-dff9aa8c51cc-oauth-serving-cert\") pod \"console-56c6f87bc5-824bh\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:30:40.369059 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.368992 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff1595e1-86db-4f53-a118-dff9aa8c51cc-service-ca\") pod \"console-56c6f87bc5-824bh\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:30:40.469585 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.469493 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff1595e1-86db-4f53-a118-dff9aa8c51cc-service-ca\") pod \"console-56c6f87bc5-824bh\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:30:40.469585 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.469559 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff1595e1-86db-4f53-a118-dff9aa8c51cc-console-serving-cert\") pod \"console-56c6f87bc5-824bh\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:30:40.469585 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.469585 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9jkg\" (UniqueName: \"kubernetes.io/projected/ff1595e1-86db-4f53-a118-dff9aa8c51cc-kube-api-access-z9jkg\") pod \"console-56c6f87bc5-824bh\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:30:40.469891 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.469622 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff1595e1-86db-4f53-a118-dff9aa8c51cc-trusted-ca-bundle\") pod \"console-56c6f87bc5-824bh\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:30:40.469891 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.469658 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff1595e1-86db-4f53-a118-dff9aa8c51cc-console-oauth-config\") pod \"console-56c6f87bc5-824bh\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:30:40.470112 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.470071 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff1595e1-86db-4f53-a118-dff9aa8c51cc-console-config\") pod \"console-56c6f87bc5-824bh\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:30:40.470195 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.470148 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff1595e1-86db-4f53-a118-dff9aa8c51cc-oauth-serving-cert\") pod \"console-56c6f87bc5-824bh\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:30:40.470974 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.470434 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff1595e1-86db-4f53-a118-dff9aa8c51cc-service-ca\") pod \"console-56c6f87bc5-824bh\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:30:40.470974 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.470686 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff1595e1-86db-4f53-a118-dff9aa8c51cc-trusted-ca-bundle\") pod \"console-56c6f87bc5-824bh\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:30:40.470974 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.470848 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff1595e1-86db-4f53-a118-dff9aa8c51cc-oauth-serving-cert\") pod \"console-56c6f87bc5-824bh\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:30:40.470974 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.470934 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff1595e1-86db-4f53-a118-dff9aa8c51cc-console-config\") pod \"console-56c6f87bc5-824bh\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:30:40.472650 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.472628 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff1595e1-86db-4f53-a118-dff9aa8c51cc-console-oauth-config\") pod \"console-56c6f87bc5-824bh\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:30:40.472810 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.472782 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff1595e1-86db-4f53-a118-dff9aa8c51cc-console-serving-cert\") pod \"console-56c6f87bc5-824bh\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:30:40.487282 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.487251 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9jkg\" (UniqueName: \"kubernetes.io/projected/ff1595e1-86db-4f53-a118-dff9aa8c51cc-kube-api-access-z9jkg\") pod \"console-56c6f87bc5-824bh\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:30:40.519103 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.519060 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:30:40.852658 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:40.852623 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56c6f87bc5-824bh"] Jul 09 13:30:40.856457 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:30:40.856432 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff1595e1_86db_4f53_a118_dff9aa8c51cc.slice/crio-592af4fe8261a2b38e13030aec00a07832f298867f104321da0b844107381dba WatchSource:0}: Error finding container 592af4fe8261a2b38e13030aec00a07832f298867f104321da0b844107381dba: Status 404 returned error can't find the container with id 592af4fe8261a2b38e13030aec00a07832f298867f104321da0b844107381dba Jul 09 13:30:41.230496 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:41.230458 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lwvt4" event={"ID":"b125e19d-1900-4c3d-945a-61de5502b779","Type":"ContainerStarted","Data":"a6247ed5359ce210fb60d51f11ad2bae46cc159bcd85cbe35ef49bbeab103ca7"} Jul 09 13:30:41.230974 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:41.230513 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lwvt4" event={"ID":"b125e19d-1900-4c3d-945a-61de5502b779","Type":"ContainerStarted","Data":"94b1793cdfc79e1cc79f40b8638708d8a2ea80c1ff2077cb02ac98a782550df8"} Jul 09 13:30:41.232085 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:41.232056 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56c6f87bc5-824bh" event={"ID":"ff1595e1-86db-4f53-a118-dff9aa8c51cc","Type":"ContainerStarted","Data":"cc7c12dfa092231fadcb76d9bd69930dce9c7f63b11fb3e7919344777c4408b7"} Jul 09 13:30:41.232212 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:41.232102 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56c6f87bc5-824bh" event={"ID":"ff1595e1-86db-4f53-a118-dff9aa8c51cc","Type":"ContainerStarted","Data":"592af4fe8261a2b38e13030aec00a07832f298867f104321da0b844107381dba"} Jul 09 13:30:41.256036 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:41.255967 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-lwvt4" podStartSLOduration=2.18455156 podStartE2EDuration="4.255950814s" totalImagesPullingTime="2.071399254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:30:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:41.254340233 +0000 UTC m=+58.759642519" watchObservedRunningTime="2026-07-09 13:30:41.255950814 +0000 UTC m=+58.761253082" Jul 09 13:30:41.279224 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:41.279177 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56c6f87bc5-824bh" podStartSLOduration=1.279159663 podStartE2EDuration="1.279159663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:30:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:41.278333838 +0000 UTC m=+58.783636108" watchObservedRunningTime="2026-07-09 13:30:41.279159663 +0000 UTC m=+58.784461930" Jul 09 13:30:42.564800 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.564752 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:30:42.565234 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.564825 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:30:42.569693 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.569668 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:30:42.943862 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.943828 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-x4jf5"] Jul 09 13:30:42.946653 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.946636 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x4jf5" Jul 09 13:30:42.949657 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.949636 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 09 13:30:42.949917 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.949893 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-md6tp\"" Jul 09 13:30:42.950174 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.950158 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 09 13:30:42.951267 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.951247 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz"] Jul 09 13:30:42.953952 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.953933 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" Jul 09 13:30:42.955441 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.955420 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-x4jf5"] Jul 09 13:30:42.956635 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.956616 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 09 13:30:42.956772 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.956750 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 09 13:30:42.956902 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.956880 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-grh7h\"" Jul 09 13:30:42.957125 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.957101 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 09 13:30:42.969383 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.969356 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz"] Jul 09 13:30:42.979216 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.979188 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-pq8wj"] Jul 09 13:30:42.981602 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.981582 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:42.984100 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.984081 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 13:30:42.984339 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.984318 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xs29z\"" Jul 09 13:30:42.984402 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.984352 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 13:30:42.984402 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.984391 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 13:30:42.992335 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.992306 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/edec525a-47f6-4c88-92c8-4e502b6f2739-node-exporter-wtmp\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:42.992446 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.992344 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/edec525a-47f6-4c88-92c8-4e502b6f2739-node-exporter-tls\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:42.992446 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.992365 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/edec525a-47f6-4c88-92c8-4e502b6f2739-root\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:42.992446 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.992407 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h64fv\" (UniqueName: \"kubernetes.io/projected/62b06276-f4ff-4d42-a305-339bf1c453bd-kube-api-access-h64fv\") pod \"kube-state-metrics-7c9dbb8b7d-ttqkz\" (UID: \"62b06276-f4ff-4d42-a305-339bf1c453bd\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" Jul 09 13:30:42.992446 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.992431 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/edec525a-47f6-4c88-92c8-4e502b6f2739-metrics-client-ca\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:42.992589 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.992458 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/62b06276-f4ff-4d42-a305-339bf1c453bd-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-ttqkz\" (UID: \"62b06276-f4ff-4d42-a305-339bf1c453bd\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" Jul 09 13:30:42.992589 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.992486 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62b06276-f4ff-4d42-a305-339bf1c453bd-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-ttqkz\" (UID: \"62b06276-f4ff-4d42-a305-339bf1c453bd\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" Jul 09 13:30:42.992589 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.992541 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/edec525a-47f6-4c88-92c8-4e502b6f2739-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:42.992589 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.992563 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/151e80f3-5a05-456b-b719-41c7d7f946c6-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-x4jf5\" (UID: \"151e80f3-5a05-456b-b719-41c7d7f946c6\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x4jf5" Jul 09 13:30:42.992589 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.992580 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/151e80f3-5a05-456b-b719-41c7d7f946c6-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-x4jf5\" (UID: \"151e80f3-5a05-456b-b719-41c7d7f946c6\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x4jf5" Jul 09 13:30:42.992824 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.992600 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/edec525a-47f6-4c88-92c8-4e502b6f2739-node-exporter-accelerators-collector-config\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:42.992824 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.992681 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/151e80f3-5a05-456b-b719-41c7d7f946c6-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-x4jf5\" (UID: \"151e80f3-5a05-456b-b719-41c7d7f946c6\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x4jf5" Jul 09 13:30:42.992824 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.992738 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/62b06276-f4ff-4d42-a305-339bf1c453bd-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-ttqkz\" (UID: \"62b06276-f4ff-4d42-a305-339bf1c453bd\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" Jul 09 13:30:42.992824 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.992773 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62b06276-f4ff-4d42-a305-339bf1c453bd-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-ttqkz\" (UID: \"62b06276-f4ff-4d42-a305-339bf1c453bd\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" Jul 09 13:30:42.992824 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.992802 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/edec525a-47f6-4c88-92c8-4e502b6f2739-node-exporter-textfile\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:42.993072 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.992840 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghb8x\" (UniqueName: \"kubernetes.io/projected/edec525a-47f6-4c88-92c8-4e502b6f2739-kube-api-access-ghb8x\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:42.993072 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.992872 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/edec525a-47f6-4c88-92c8-4e502b6f2739-sys\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:42.993072 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.992910 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vft9h\" (UniqueName: \"kubernetes.io/projected/151e80f3-5a05-456b-b719-41c7d7f946c6-kube-api-access-vft9h\") pod \"openshift-state-metrics-54cf96dfcf-x4jf5\" (UID: \"151e80f3-5a05-456b-b719-41c7d7f946c6\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x4jf5" Jul 09 13:30:42.993072 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:42.992940 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/62b06276-f4ff-4d42-a305-339bf1c453bd-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-ttqkz\" (UID: \"62b06276-f4ff-4d42-a305-339bf1c453bd\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" Jul 09 13:30:43.094332 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.094288 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62b06276-f4ff-4d42-a305-339bf1c453bd-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-ttqkz\" (UID: \"62b06276-f4ff-4d42-a305-339bf1c453bd\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" Jul 09 13:30:43.094332 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.094336 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/edec525a-47f6-4c88-92c8-4e502b6f2739-node-exporter-textfile\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:43.094574 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.094379 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghb8x\" (UniqueName: \"kubernetes.io/projected/edec525a-47f6-4c88-92c8-4e502b6f2739-kube-api-access-ghb8x\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:43.094574 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.094411 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/edec525a-47f6-4c88-92c8-4e502b6f2739-sys\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:43.094574 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.094446 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vft9h\" (UniqueName: \"kubernetes.io/projected/151e80f3-5a05-456b-b719-41c7d7f946c6-kube-api-access-vft9h\") pod \"openshift-state-metrics-54cf96dfcf-x4jf5\" (UID: \"151e80f3-5a05-456b-b719-41c7d7f946c6\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x4jf5" Jul 09 13:30:43.094574 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.094476 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/62b06276-f4ff-4d42-a305-339bf1c453bd-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-ttqkz\" (UID: \"62b06276-f4ff-4d42-a305-339bf1c453bd\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" Jul 09 13:30:43.094574 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.094526 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/edec525a-47f6-4c88-92c8-4e502b6f2739-sys\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:43.094793 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.094602 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/edec525a-47f6-4c88-92c8-4e502b6f2739-node-exporter-wtmp\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:43.094793 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.094653 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/edec525a-47f6-4c88-92c8-4e502b6f2739-node-exporter-tls\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:43.094793 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:43.094741 2573 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 09 13:30:43.094793 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.094757 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/edec525a-47f6-4c88-92c8-4e502b6f2739-root\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:43.094962 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.094799 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h64fv\" (UniqueName: \"kubernetes.io/projected/62b06276-f4ff-4d42-a305-339bf1c453bd-kube-api-access-h64fv\") pod \"kube-state-metrics-7c9dbb8b7d-ttqkz\" (UID: \"62b06276-f4ff-4d42-a305-339bf1c453bd\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" Jul 09 13:30:43.094962 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.094802 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/edec525a-47f6-4c88-92c8-4e502b6f2739-node-exporter-wtmp\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:43.094962 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.094827 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/62b06276-f4ff-4d42-a305-339bf1c453bd-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-ttqkz\" (UID: \"62b06276-f4ff-4d42-a305-339bf1c453bd\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" Jul 09 13:30:43.094962 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:43.094816 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/edec525a-47f6-4c88-92c8-4e502b6f2739-node-exporter-tls podName:edec525a-47f6-4c88-92c8-4e502b6f2739 nodeName:}" failed. No retries permitted until 2026-07-09 13:30:43.594795595 +0000 UTC m=+61.100097841 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/edec525a-47f6-4c88-92c8-4e502b6f2739-node-exporter-tls") pod "node-exporter-pq8wj" (UID: "edec525a-47f6-4c88-92c8-4e502b6f2739") : secret "node-exporter-tls" not found Jul 09 13:30:43.094962 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.094764 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/edec525a-47f6-4c88-92c8-4e502b6f2739-node-exporter-textfile\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:43.094962 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.094866 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/edec525a-47f6-4c88-92c8-4e502b6f2739-root\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:43.094962 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.094907 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/edec525a-47f6-4c88-92c8-4e502b6f2739-metrics-client-ca\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:43.094962 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.094938 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/62b06276-f4ff-4d42-a305-339bf1c453bd-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-ttqkz\" (UID: \"62b06276-f4ff-4d42-a305-339bf1c453bd\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" Jul 09 13:30:43.094962 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.094959 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62b06276-f4ff-4d42-a305-339bf1c453bd-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-ttqkz\" (UID: \"62b06276-f4ff-4d42-a305-339bf1c453bd\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" Jul 09 13:30:43.095467 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.095002 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/edec525a-47f6-4c88-92c8-4e502b6f2739-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:43.095467 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:43.095047 2573 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 09 13:30:43.095467 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.095060 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/151e80f3-5a05-456b-b719-41c7d7f946c6-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-x4jf5\" (UID: \"151e80f3-5a05-456b-b719-41c7d7f946c6\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x4jf5" Jul 09 13:30:43.095467 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.095090 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/151e80f3-5a05-456b-b719-41c7d7f946c6-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-x4jf5\" (UID: \"151e80f3-5a05-456b-b719-41c7d7f946c6\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x4jf5" Jul 09 13:30:43.095467 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:43.095103 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62b06276-f4ff-4d42-a305-339bf1c453bd-kube-state-metrics-tls podName:62b06276-f4ff-4d42-a305-339bf1c453bd nodeName:}" failed. No retries permitted until 2026-07-09 13:30:43.595087438 +0000 UTC m=+61.100389698 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/62b06276-f4ff-4d42-a305-339bf1c453bd-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-ttqkz" (UID: "62b06276-f4ff-4d42-a305-339bf1c453bd") : secret "kube-state-metrics-tls" not found Jul 09 13:30:43.095467 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.095130 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/edec525a-47f6-4c88-92c8-4e502b6f2739-node-exporter-accelerators-collector-config\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:43.095467 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.095181 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62b06276-f4ff-4d42-a305-339bf1c453bd-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-ttqkz\" (UID: \"62b06276-f4ff-4d42-a305-339bf1c453bd\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" Jul 09 13:30:43.095467 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:43.095261 2573 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 09 13:30:43.095467 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.095299 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/151e80f3-5a05-456b-b719-41c7d7f946c6-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-x4jf5\" (UID: \"151e80f3-5a05-456b-b719-41c7d7f946c6\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x4jf5" Jul 09 13:30:43.095467 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:43.095319 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/151e80f3-5a05-456b-b719-41c7d7f946c6-openshift-state-metrics-tls podName:151e80f3-5a05-456b-b719-41c7d7f946c6 nodeName:}" failed. No retries permitted until 2026-07-09 13:30:43.595303332 +0000 UTC m=+61.100605590 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/151e80f3-5a05-456b-b719-41c7d7f946c6-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-x4jf5" (UID: "151e80f3-5a05-456b-b719-41c7d7f946c6") : secret "openshift-state-metrics-tls" not found Jul 09 13:30:43.095467 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.095372 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/62b06276-f4ff-4d42-a305-339bf1c453bd-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-ttqkz\" (UID: \"62b06276-f4ff-4d42-a305-339bf1c453bd\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" Jul 09 13:30:43.095990 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.095550 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/edec525a-47f6-4c88-92c8-4e502b6f2739-metrics-client-ca\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:43.095990 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.095663 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/edec525a-47f6-4c88-92c8-4e502b6f2739-node-exporter-accelerators-collector-config\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:43.096119 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.096036 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/62b06276-f4ff-4d42-a305-339bf1c453bd-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-ttqkz\" (UID: \"62b06276-f4ff-4d42-a305-339bf1c453bd\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" Jul 09 13:30:43.096119 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.095994 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/151e80f3-5a05-456b-b719-41c7d7f946c6-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-x4jf5\" (UID: \"151e80f3-5a05-456b-b719-41c7d7f946c6\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x4jf5" Jul 09 13:30:43.097583 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.097550 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/edec525a-47f6-4c88-92c8-4e502b6f2739-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:43.097692 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.097614 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62b06276-f4ff-4d42-a305-339bf1c453bd-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-ttqkz\" (UID: \"62b06276-f4ff-4d42-a305-339bf1c453bd\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" Jul 09 13:30:43.097692 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.097632 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/151e80f3-5a05-456b-b719-41c7d7f946c6-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-x4jf5\" (UID: \"151e80f3-5a05-456b-b719-41c7d7f946c6\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x4jf5" Jul 09 13:30:43.113256 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.113220 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghb8x\" (UniqueName: \"kubernetes.io/projected/edec525a-47f6-4c88-92c8-4e502b6f2739-kube-api-access-ghb8x\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:43.113402 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.113258 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h64fv\" (UniqueName: \"kubernetes.io/projected/62b06276-f4ff-4d42-a305-339bf1c453bd-kube-api-access-h64fv\") pod \"kube-state-metrics-7c9dbb8b7d-ttqkz\" (UID: \"62b06276-f4ff-4d42-a305-339bf1c453bd\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" Jul 09 13:30:43.114341 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.114323 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vft9h\" (UniqueName: \"kubernetes.io/projected/151e80f3-5a05-456b-b719-41c7d7f946c6-kube-api-access-vft9h\") pod \"openshift-state-metrics-54cf96dfcf-x4jf5\" (UID: \"151e80f3-5a05-456b-b719-41c7d7f946c6\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x4jf5" Jul 09 13:30:43.129670 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.129643 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vmms8" Jul 09 13:30:43.243606 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.243528 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:30:43.599656 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.599570 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/edec525a-47f6-4c88-92c8-4e502b6f2739-node-exporter-tls\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:43.599656 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.599609 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/62b06276-f4ff-4d42-a305-339bf1c453bd-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-ttqkz\" (UID: \"62b06276-f4ff-4d42-a305-339bf1c453bd\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" Jul 09 13:30:43.599656 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.599639 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/151e80f3-5a05-456b-b719-41c7d7f946c6-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-x4jf5\" (UID: \"151e80f3-5a05-456b-b719-41c7d7f946c6\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x4jf5" Jul 09 13:30:43.601935 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.601903 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/edec525a-47f6-4c88-92c8-4e502b6f2739-node-exporter-tls\") pod \"node-exporter-pq8wj\" (UID: \"edec525a-47f6-4c88-92c8-4e502b6f2739\") " pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:43.602176 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.602153 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/151e80f3-5a05-456b-b719-41c7d7f946c6-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-x4jf5\" (UID: \"151e80f3-5a05-456b-b719-41c7d7f946c6\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x4jf5" Jul 09 13:30:43.602176 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.602166 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/62b06276-f4ff-4d42-a305-339bf1c453bd-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-ttqkz\" (UID: \"62b06276-f4ff-4d42-a305-339bf1c453bd\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" Jul 09 13:30:43.857974 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.857879 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x4jf5" Jul 09 13:30:43.866090 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.866061 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" Jul 09 13:30:43.890262 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:43.890235 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pq8wj" Jul 09 13:30:43.915810 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:30:43.915744 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedec525a_47f6_4c88_92c8_4e502b6f2739.slice/crio-40c7ad3275d6e99ea9cb2a31acad3cc4219f73dfdca35e70439bc0cbc4e5641d WatchSource:0}: Error finding container 40c7ad3275d6e99ea9cb2a31acad3cc4219f73dfdca35e70439bc0cbc4e5641d: Status 404 returned error can't find the container with id 40c7ad3275d6e99ea9cb2a31acad3cc4219f73dfdca35e70439bc0cbc4e5641d Jul 09 13:30:44.066838 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.066809 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-x4jf5"] Jul 09 13:30:44.066838 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:30:44.066821 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62b06276_f4ff_4d42_a305_339bf1c453bd.slice/crio-1fe58b74791a69234611bf8c48e975637b70a1d631fdc28d9397ae586a13f21d WatchSource:0}: Error finding container 1fe58b74791a69234611bf8c48e975637b70a1d631fdc28d9397ae586a13f21d: Status 404 returned error can't find the container with id 1fe58b74791a69234611bf8c48e975637b70a1d631fdc28d9397ae586a13f21d Jul 09 13:30:44.067331 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:30:44.067313 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod151e80f3_5a05_456b_b719_41c7d7f946c6.slice/crio-934d0685c7661bbcb93ddadc1732af897191a849cd77b016cb8908d189ddd517 WatchSource:0}: Error finding container 934d0685c7661bbcb93ddadc1732af897191a849cd77b016cb8908d189ddd517: Status 404 returned error can't find the container with id 934d0685c7661bbcb93ddadc1732af897191a849cd77b016cb8908d189ddd517 Jul 09 13:30:44.067823 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.067800 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz"] Jul 09 13:30:44.219625 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.219591 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 13:30:44.248076 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.248045 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 13:30:44.248232 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.248129 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.248232 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.248145 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pq8wj" event={"ID":"edec525a-47f6-4c88-92c8-4e502b6f2739","Type":"ContainerStarted","Data":"40c7ad3275d6e99ea9cb2a31acad3cc4219f73dfdca35e70439bc0cbc4e5641d"} Jul 09 13:30:44.248232 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.248185 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" event={"ID":"62b06276-f4ff-4d42-a305-339bf1c453bd","Type":"ContainerStarted","Data":"1fe58b74791a69234611bf8c48e975637b70a1d631fdc28d9397ae586a13f21d"} Jul 09 13:30:44.248232 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.248218 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x4jf5" event={"ID":"151e80f3-5a05-456b-b719-41c7d7f946c6","Type":"ContainerStarted","Data":"8107fe8629ac16395ebc5d516a968b9254752d2d66a985612a68a1f7b4be7f15"} Jul 09 13:30:44.248452 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.248249 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x4jf5" event={"ID":"151e80f3-5a05-456b-b719-41c7d7f946c6","Type":"ContainerStarted","Data":"934d0685c7661bbcb93ddadc1732af897191a849cd77b016cb8908d189ddd517"} Jul 09 13:30:44.251618 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.251583 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 09 13:30:44.251775 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.251621 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 09 13:30:44.251775 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.251665 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 09 13:30:44.251775 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.251584 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 09 13:30:44.252006 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.251988 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 09 13:30:44.252098 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.251997 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 09 13:30:44.252149 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.252128 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 09 13:30:44.252798 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.252778 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 09 13:30:44.253345 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.253326 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-mbtzb\"" Jul 09 13:30:44.272521 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.268908 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 09 13:30:44.305814 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.305780 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-config-volume\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.306406 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.305857 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.306406 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.305885 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-web-config\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.306406 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.305908 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b87b16cf-44bd-4ef7-99b3-cf305030612c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.306406 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.305936 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b87b16cf-44bd-4ef7-99b3-cf305030612c-config-out\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.306406 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.305972 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b87b16cf-44bd-4ef7-99b3-cf305030612c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.306406 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.306087 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.306406 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.306133 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.306406 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.306176 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgc82\" (UniqueName: \"kubernetes.io/projected/b87b16cf-44bd-4ef7-99b3-cf305030612c-kube-api-access-sgc82\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.306406 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.306289 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.306406 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.306325 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b87b16cf-44bd-4ef7-99b3-cf305030612c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.306901 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.306476 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b87b16cf-44bd-4ef7-99b3-cf305030612c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.306901 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.306514 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.407510 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.407472 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-config-volume\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.407722 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.407555 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.407722 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.407583 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-web-config\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.407722 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.407605 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b87b16cf-44bd-4ef7-99b3-cf305030612c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.407722 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.407633 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b87b16cf-44bd-4ef7-99b3-cf305030612c-config-out\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.407932 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.407854 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b87b16cf-44bd-4ef7-99b3-cf305030612c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.407932 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.407896 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.408044 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.407931 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.408044 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.407976 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgc82\" (UniqueName: \"kubernetes.io/projected/b87b16cf-44bd-4ef7-99b3-cf305030612c-kube-api-access-sgc82\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.408196 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.408157 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.408332 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.408253 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b87b16cf-44bd-4ef7-99b3-cf305030612c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.408332 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.408274 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b87b16cf-44bd-4ef7-99b3-cf305030612c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.408445 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.408351 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b87b16cf-44bd-4ef7-99b3-cf305030612c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.408445 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.408386 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.409816 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.409715 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b87b16cf-44bd-4ef7-99b3-cf305030612c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.410085 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.410062 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b87b16cf-44bd-4ef7-99b3-cf305030612c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.411061 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.410993 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.412294 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.412267 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.413244 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.413226 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.413892 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.413643 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.413892 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.413714 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b87b16cf-44bd-4ef7-99b3-cf305030612c-config-out\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.413892 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.413732 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-web-config\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.413892 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.413839 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-config-volume\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.414128 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.413962 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b87b16cf-44bd-4ef7-99b3-cf305030612c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.415125 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.415086 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.420582 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.420560 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgc82\" (UniqueName: \"kubernetes.io/projected/b87b16cf-44bd-4ef7-99b3-cf305030612c-kube-api-access-sgc82\") pod \"alertmanager-main-0\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.570071 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.570029 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:30:44.779200 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:44.779169 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 13:30:44.967014 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:30:44.966932 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb87b16cf_44bd_4ef7_99b3_cf305030612c.slice/crio-8382057708fd427098f416665f8766b5965e319595ddc60fbf1fe33f597bf9b2 WatchSource:0}: Error finding container 8382057708fd427098f416665f8766b5965e319595ddc60fbf1fe33f597bf9b2: Status 404 returned error can't find the container with id 8382057708fd427098f416665f8766b5965e319595ddc60fbf1fe33f597bf9b2 Jul 09 13:30:45.045555 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.045523 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6d5688b76d-th79t"] Jul 09 13:30:45.053616 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.053585 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.056676 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.056651 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 09 13:30:45.056827 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.056700 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 09 13:30:45.056827 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.056651 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 09 13:30:45.057008 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.056987 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-bccxk\"" Jul 09 13:30:45.057195 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.057182 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 09 13:30:45.057382 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.057365 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 09 13:30:45.057482 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.057439 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-5tabsjbq9dto9\"" Jul 09 13:30:45.064114 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.064089 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6d5688b76d-th79t"] Jul 09 13:30:45.116002 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.115971 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b2d57361-d50a-47a0-9f50-d308478d019a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d5688b76d-th79t\" (UID: \"b2d57361-d50a-47a0-9f50-d308478d019a\") " pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.116158 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.116037 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b2d57361-d50a-47a0-9f50-d308478d019a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d5688b76d-th79t\" (UID: \"b2d57361-d50a-47a0-9f50-d308478d019a\") " pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.116158 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.116061 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b2d57361-d50a-47a0-9f50-d308478d019a-secret-grpc-tls\") pod \"thanos-querier-6d5688b76d-th79t\" (UID: \"b2d57361-d50a-47a0-9f50-d308478d019a\") " pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.116158 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.116129 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b2d57361-d50a-47a0-9f50-d308478d019a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d5688b76d-th79t\" (UID: \"b2d57361-d50a-47a0-9f50-d308478d019a\") " pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.116351 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.116219 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b2d57361-d50a-47a0-9f50-d308478d019a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d5688b76d-th79t\" (UID: \"b2d57361-d50a-47a0-9f50-d308478d019a\") " pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.116351 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.116243 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2d57361-d50a-47a0-9f50-d308478d019a-metrics-client-ca\") pod \"thanos-querier-6d5688b76d-th79t\" (UID: \"b2d57361-d50a-47a0-9f50-d308478d019a\") " pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.116351 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.116294 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b2d57361-d50a-47a0-9f50-d308478d019a-secret-thanos-querier-tls\") pod \"thanos-querier-6d5688b76d-th79t\" (UID: \"b2d57361-d50a-47a0-9f50-d308478d019a\") " pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.116351 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.116319 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhs54\" (UniqueName: \"kubernetes.io/projected/b2d57361-d50a-47a0-9f50-d308478d019a-kube-api-access-zhs54\") pod \"thanos-querier-6d5688b76d-th79t\" (UID: \"b2d57361-d50a-47a0-9f50-d308478d019a\") " pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.206254 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.206223 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9bhjn" Jul 09 13:30:45.216912 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.216885 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b2d57361-d50a-47a0-9f50-d308478d019a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d5688b76d-th79t\" (UID: \"b2d57361-d50a-47a0-9f50-d308478d019a\") " pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.217130 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.216915 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2d57361-d50a-47a0-9f50-d308478d019a-metrics-client-ca\") pod \"thanos-querier-6d5688b76d-th79t\" (UID: \"b2d57361-d50a-47a0-9f50-d308478d019a\") " pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.217130 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.217039 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b2d57361-d50a-47a0-9f50-d308478d019a-secret-thanos-querier-tls\") pod \"thanos-querier-6d5688b76d-th79t\" (UID: \"b2d57361-d50a-47a0-9f50-d308478d019a\") " pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.217130 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.217091 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhs54\" (UniqueName: \"kubernetes.io/projected/b2d57361-d50a-47a0-9f50-d308478d019a-kube-api-access-zhs54\") pod \"thanos-querier-6d5688b76d-th79t\" (UID: \"b2d57361-d50a-47a0-9f50-d308478d019a\") " pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.217307 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.217140 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b2d57361-d50a-47a0-9f50-d308478d019a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d5688b76d-th79t\" (UID: \"b2d57361-d50a-47a0-9f50-d308478d019a\") " pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.217307 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.217197 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b2d57361-d50a-47a0-9f50-d308478d019a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d5688b76d-th79t\" (UID: \"b2d57361-d50a-47a0-9f50-d308478d019a\") " pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.217307 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.217225 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b2d57361-d50a-47a0-9f50-d308478d019a-secret-grpc-tls\") pod \"thanos-querier-6d5688b76d-th79t\" (UID: \"b2d57361-d50a-47a0-9f50-d308478d019a\") " pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.217307 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.217268 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b2d57361-d50a-47a0-9f50-d308478d019a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d5688b76d-th79t\" (UID: \"b2d57361-d50a-47a0-9f50-d308478d019a\") " pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.217797 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.217690 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2d57361-d50a-47a0-9f50-d308478d019a-metrics-client-ca\") pod \"thanos-querier-6d5688b76d-th79t\" (UID: \"b2d57361-d50a-47a0-9f50-d308478d019a\") " pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.220456 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.220402 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b2d57361-d50a-47a0-9f50-d308478d019a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d5688b76d-th79t\" (UID: \"b2d57361-d50a-47a0-9f50-d308478d019a\") " pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.220456 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.220444 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b2d57361-d50a-47a0-9f50-d308478d019a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d5688b76d-th79t\" (UID: \"b2d57361-d50a-47a0-9f50-d308478d019a\") " pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.220663 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.220642 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b2d57361-d50a-47a0-9f50-d308478d019a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d5688b76d-th79t\" (UID: \"b2d57361-d50a-47a0-9f50-d308478d019a\") " pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.221462 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.221436 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b2d57361-d50a-47a0-9f50-d308478d019a-secret-thanos-querier-tls\") pod \"thanos-querier-6d5688b76d-th79t\" (UID: \"b2d57361-d50a-47a0-9f50-d308478d019a\") " pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.221639 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.221612 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b2d57361-d50a-47a0-9f50-d308478d019a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d5688b76d-th79t\" (UID: \"b2d57361-d50a-47a0-9f50-d308478d019a\") " pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.221754 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.221731 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b2d57361-d50a-47a0-9f50-d308478d019a-secret-grpc-tls\") pod \"thanos-querier-6d5688b76d-th79t\" (UID: \"b2d57361-d50a-47a0-9f50-d308478d019a\") " pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.232840 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.232797 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhs54\" (UniqueName: \"kubernetes.io/projected/b2d57361-d50a-47a0-9f50-d308478d019a-kube-api-access-zhs54\") pod \"thanos-querier-6d5688b76d-th79t\" (UID: \"b2d57361-d50a-47a0-9f50-d308478d019a\") " pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.254988 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.254938 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x4jf5" event={"ID":"151e80f3-5a05-456b-b719-41c7d7f946c6","Type":"ContainerStarted","Data":"b238ef151c5e807fc4a8b1c813db40124f217ac3619e639980005df25cbd29b0"} Jul 09 13:30:45.256543 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.256455 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b87b16cf-44bd-4ef7-99b3-cf305030612c","Type":"ContainerStarted","Data":"8382057708fd427098f416665f8766b5965e319595ddc60fbf1fe33f597bf9b2"} Jul 09 13:30:45.366392 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.366359 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:45.658115 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:45.658078 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6d5688b76d-th79t"] Jul 09 13:30:45.773941 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:30:45.773835 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2d57361_d50a_47a0_9f50_d308478d019a.slice/crio-e5adc1902cf302873bd65b397be1f9de9bbd639a31db6d6ba748ca7946efad95 WatchSource:0}: Error finding container e5adc1902cf302873bd65b397be1f9de9bbd639a31db6d6ba748ca7946efad95: Status 404 returned error can't find the container with id e5adc1902cf302873bd65b397be1f9de9bbd639a31db6d6ba748ca7946efad95 Jul 09 13:30:46.266812 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:46.266775 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" event={"ID":"62b06276-f4ff-4d42-a305-339bf1c453bd","Type":"ContainerStarted","Data":"cdea551e13923a9f7fdc028a10225a76e86d0020847add2e7e0b3696fbbf9f6d"} Jul 09 13:30:46.267253 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:46.266851 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" event={"ID":"62b06276-f4ff-4d42-a305-339bf1c453bd","Type":"ContainerStarted","Data":"e342b5f1d27e8ed4fdf86092253f9ef35c2f0794667afde803d7153e82893b8b"} Jul 09 13:30:46.273737 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:46.273694 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x4jf5" event={"ID":"151e80f3-5a05-456b-b719-41c7d7f946c6","Type":"ContainerStarted","Data":"2051263e849a8aa7eccef1fc3059e554e66ff9c71446f6b8d7e64c2f7b205bd7"} Jul 09 13:30:46.275725 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:46.275694 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pq8wj" event={"ID":"edec525a-47f6-4c88-92c8-4e502b6f2739","Type":"ContainerStarted","Data":"7b94fcf081256ca390d194a1514623193d6b468774e91048640332ade7ddd35d"} Jul 09 13:30:46.277214 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:46.277189 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" event={"ID":"b2d57361-d50a-47a0-9f50-d308478d019a","Type":"ContainerStarted","Data":"e5adc1902cf302873bd65b397be1f9de9bbd639a31db6d6ba748ca7946efad95"} Jul 09 13:30:46.332048 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:46.331642 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-x4jf5" podStartSLOduration=2.82481508 podStartE2EDuration="4.331618772s" totalImagesPullingTime="1.506803692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:46.300523315 +0000 UTC m=+63.805825602" watchObservedRunningTime="2026-07-09 13:30:46.331618772 +0000 UTC m=+63.836921035" Jul 09 13:30:47.283419 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.283377 2573 generic.go:356] "Generic (PLEG): container finished" podID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerID="72978e0a2c65aa9a24844d253eef0c8ff4645cbd9e4491a876882252b6617d5e" exitCode=0 Jul 09 13:30:47.283879 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.283514 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b87b16cf-44bd-4ef7-99b3-cf305030612c","Type":"ContainerDied","Data":"72978e0a2c65aa9a24844d253eef0c8ff4645cbd9e4491a876882252b6617d5e"} Jul 09 13:30:47.285547 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.285281 2573 generic.go:356] "Generic (PLEG): container finished" podID="edec525a-47f6-4c88-92c8-4e502b6f2739" containerID="7b94fcf081256ca390d194a1514623193d6b468774e91048640332ade7ddd35d" exitCode=0 Jul 09 13:30:47.285547 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.285355 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pq8wj" event={"ID":"edec525a-47f6-4c88-92c8-4e502b6f2739","Type":"ContainerDied","Data":"7b94fcf081256ca390d194a1514623193d6b468774e91048640332ade7ddd35d"} Jul 09 13:30:47.287801 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.287777 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" event={"ID":"62b06276-f4ff-4d42-a305-339bf1c453bd","Type":"ContainerStarted","Data":"21d2471db1058f89a3d68093086a63327386083802238fe483455a77ce5c62d5"} Jul 09 13:30:47.373651 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.373541 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ttqkz" podStartSLOduration=3.66405495 podStartE2EDuration="5.37351912s" totalImagesPullingTime="1.70946417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:47.371959811 +0000 UTC m=+64.877262079" watchObservedRunningTime="2026-07-09 13:30:47.37351912 +0000 UTC m=+64.878821389" Jul 09 13:30:47.441160 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.441129 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-77764697c6-kj8fm"] Jul 09 13:30:47.444330 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.444308 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:30:47.447567 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.447517 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-pj46k\"" Jul 09 13:30:47.447567 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.447540 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 09 13:30:47.448060 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.447629 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 09 13:30:47.448060 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.447544 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-7f9hmcotgjqse\"" Jul 09 13:30:47.448060 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.447544 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 09 13:30:47.448060 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.447931 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 09 13:30:47.455328 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.455264 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-77764697c6-kj8fm"] Jul 09 13:30:47.540741 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.540706 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/43a7ad9e-a15f-46d4-8acb-791cd9a2300b-metrics-server-audit-profiles\") pod \"metrics-server-77764697c6-kj8fm\" (UID: \"43a7ad9e-a15f-46d4-8acb-791cd9a2300b\") " pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:30:47.540910 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.540775 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/43a7ad9e-a15f-46d4-8acb-791cd9a2300b-secret-metrics-server-client-certs\") pod \"metrics-server-77764697c6-kj8fm\" (UID: \"43a7ad9e-a15f-46d4-8acb-791cd9a2300b\") " pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:30:47.540910 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.540845 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/43a7ad9e-a15f-46d4-8acb-791cd9a2300b-audit-log\") pod \"metrics-server-77764697c6-kj8fm\" (UID: \"43a7ad9e-a15f-46d4-8acb-791cd9a2300b\") " pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:30:47.540910 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.540905 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/43a7ad9e-a15f-46d4-8acb-791cd9a2300b-secret-metrics-server-tls\") pod \"metrics-server-77764697c6-kj8fm\" (UID: \"43a7ad9e-a15f-46d4-8acb-791cd9a2300b\") " pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:30:47.541084 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.540959 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/43a7ad9e-a15f-46d4-8acb-791cd9a2300b-client-ca-bundle\") pod \"metrics-server-77764697c6-kj8fm\" (UID: \"43a7ad9e-a15f-46d4-8acb-791cd9a2300b\") " pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:30:47.541084 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.541070 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p9jx\" (UniqueName: \"kubernetes.io/projected/43a7ad9e-a15f-46d4-8acb-791cd9a2300b-kube-api-access-7p9jx\") pod \"metrics-server-77764697c6-kj8fm\" (UID: \"43a7ad9e-a15f-46d4-8acb-791cd9a2300b\") " pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:30:47.541164 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.541148 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43a7ad9e-a15f-46d4-8acb-791cd9a2300b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-77764697c6-kj8fm\" (UID: \"43a7ad9e-a15f-46d4-8acb-791cd9a2300b\") " pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:30:47.642376 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.642337 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/43a7ad9e-a15f-46d4-8acb-791cd9a2300b-metrics-server-audit-profiles\") pod \"metrics-server-77764697c6-kj8fm\" (UID: \"43a7ad9e-a15f-46d4-8acb-791cd9a2300b\") " pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:30:47.642588 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.642405 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/43a7ad9e-a15f-46d4-8acb-791cd9a2300b-secret-metrics-server-client-certs\") pod \"metrics-server-77764697c6-kj8fm\" (UID: \"43a7ad9e-a15f-46d4-8acb-791cd9a2300b\") " pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:30:47.642588 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.642437 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/43a7ad9e-a15f-46d4-8acb-791cd9a2300b-audit-log\") pod \"metrics-server-77764697c6-kj8fm\" (UID: \"43a7ad9e-a15f-46d4-8acb-791cd9a2300b\") " pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:30:47.642588 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.642474 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/43a7ad9e-a15f-46d4-8acb-791cd9a2300b-secret-metrics-server-tls\") pod \"metrics-server-77764697c6-kj8fm\" (UID: \"43a7ad9e-a15f-46d4-8acb-791cd9a2300b\") " pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:30:47.642588 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.642506 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/43a7ad9e-a15f-46d4-8acb-791cd9a2300b-client-ca-bundle\") pod \"metrics-server-77764697c6-kj8fm\" (UID: \"43a7ad9e-a15f-46d4-8acb-791cd9a2300b\") " pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:30:47.642588 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.642576 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p9jx\" (UniqueName: \"kubernetes.io/projected/43a7ad9e-a15f-46d4-8acb-791cd9a2300b-kube-api-access-7p9jx\") pod \"metrics-server-77764697c6-kj8fm\" (UID: \"43a7ad9e-a15f-46d4-8acb-791cd9a2300b\") " pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:30:47.642901 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.642621 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43a7ad9e-a15f-46d4-8acb-791cd9a2300b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-77764697c6-kj8fm\" (UID: \"43a7ad9e-a15f-46d4-8acb-791cd9a2300b\") " pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:30:47.643313 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.643262 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/43a7ad9e-a15f-46d4-8acb-791cd9a2300b-audit-log\") pod \"metrics-server-77764697c6-kj8fm\" (UID: \"43a7ad9e-a15f-46d4-8acb-791cd9a2300b\") " pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:30:47.643442 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.643314 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43a7ad9e-a15f-46d4-8acb-791cd9a2300b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-77764697c6-kj8fm\" (UID: \"43a7ad9e-a15f-46d4-8acb-791cd9a2300b\") " pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:30:47.643622 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.643591 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/43a7ad9e-a15f-46d4-8acb-791cd9a2300b-metrics-server-audit-profiles\") pod \"metrics-server-77764697c6-kj8fm\" (UID: \"43a7ad9e-a15f-46d4-8acb-791cd9a2300b\") " pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:30:47.645430 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.645408 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/43a7ad9e-a15f-46d4-8acb-791cd9a2300b-client-ca-bundle\") pod \"metrics-server-77764697c6-kj8fm\" (UID: \"43a7ad9e-a15f-46d4-8acb-791cd9a2300b\") " pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:30:47.645636 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.645616 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/43a7ad9e-a15f-46d4-8acb-791cd9a2300b-secret-metrics-server-tls\") pod \"metrics-server-77764697c6-kj8fm\" (UID: \"43a7ad9e-a15f-46d4-8acb-791cd9a2300b\") " pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:30:47.645714 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.645674 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/43a7ad9e-a15f-46d4-8acb-791cd9a2300b-secret-metrics-server-client-certs\") pod \"metrics-server-77764697c6-kj8fm\" (UID: \"43a7ad9e-a15f-46d4-8acb-791cd9a2300b\") " pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:30:47.656897 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.656868 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p9jx\" (UniqueName: \"kubernetes.io/projected/43a7ad9e-a15f-46d4-8acb-791cd9a2300b-kube-api-access-7p9jx\") pod \"metrics-server-77764697c6-kj8fm\" (UID: \"43a7ad9e-a15f-46d4-8acb-791cd9a2300b\") " pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:30:47.709966 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.709928 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-qblcv"] Jul 09 13:30:47.714354 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.714323 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qblcv" Jul 09 13:30:47.717318 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.717290 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-fwskl\"" Jul 09 13:30:47.717465 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.717299 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 09 13:30:47.722520 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.722491 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-qblcv"] Jul 09 13:30:47.743723 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.743689 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2c41244e-abc4-4b1f-8c04-3d74077d0e6e-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-qblcv\" (UID: \"2c41244e-abc4-4b1f-8c04-3d74077d0e6e\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qblcv" Jul 09 13:30:47.743908 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.743745 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgjhh\" (UniqueName: \"kubernetes.io/projected/2c41244e-abc4-4b1f-8c04-3d74077d0e6e-kube-api-access-xgjhh\") pod \"monitoring-plugin-d5dcff7cf-qblcv\" (UID: \"2c41244e-abc4-4b1f-8c04-3d74077d0e6e\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qblcv" Jul 09 13:30:47.755920 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.755882 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:30:47.844675 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.844631 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2c41244e-abc4-4b1f-8c04-3d74077d0e6e-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-qblcv\" (UID: \"2c41244e-abc4-4b1f-8c04-3d74077d0e6e\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qblcv" Jul 09 13:30:47.844845 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.844684 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgjhh\" (UniqueName: \"kubernetes.io/projected/2c41244e-abc4-4b1f-8c04-3d74077d0e6e-kube-api-access-xgjhh\") pod \"monitoring-plugin-d5dcff7cf-qblcv\" (UID: \"2c41244e-abc4-4b1f-8c04-3d74077d0e6e\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qblcv" Jul 09 13:30:47.844845 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:47.844788 2573 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 09 13:30:47.844967 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:47.844859 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c41244e-abc4-4b1f-8c04-3d74077d0e6e-monitoring-plugin-cert podName:2c41244e-abc4-4b1f-8c04-3d74077d0e6e nodeName:}" failed. No retries permitted until 2026-07-09 13:30:48.344842734 +0000 UTC m=+65.850144980 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/2c41244e-abc4-4b1f-8c04-3d74077d0e6e-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-qblcv" (UID: "2c41244e-abc4-4b1f-8c04-3d74077d0e6e") : secret "monitoring-plugin-cert" not found Jul 09 13:30:47.854971 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.854919 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgjhh\" (UniqueName: \"kubernetes.io/projected/2c41244e-abc4-4b1f-8c04-3d74077d0e6e-kube-api-access-xgjhh\") pod \"monitoring-plugin-d5dcff7cf-qblcv\" (UID: \"2c41244e-abc4-4b1f-8c04-3d74077d0e6e\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qblcv" Jul 09 13:30:47.961793 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:47.961685 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-77764697c6-kj8fm"] Jul 09 13:30:47.966247 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:30:47.966220 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43a7ad9e_a15f_46d4_8acb_791cd9a2300b.slice/crio-3fcae5bdee6eac226cba4971ec83cb7cb99d9a8afe6fc39e67478fd91ad15625 WatchSource:0}: Error finding container 3fcae5bdee6eac226cba4971ec83cb7cb99d9a8afe6fc39e67478fd91ad15625: Status 404 returned error can't find the container with id 3fcae5bdee6eac226cba4971ec83cb7cb99d9a8afe6fc39e67478fd91ad15625 Jul 09 13:30:48.210435 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.210283 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7b57485f5f-89v49"] Jul 09 13:30:48.213259 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.213228 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.216268 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.216225 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-kgv5d\"" Jul 09 13:30:48.216268 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.216241 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 09 13:30:48.216520 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.216371 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 09 13:30:48.216520 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.216502 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 09 13:30:48.216634 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.216561 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 09 13:30:48.216800 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.216786 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 09 13:30:48.223103 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.222863 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 09 13:30:48.227703 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.227673 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7b57485f5f-89v49"] Jul 09 13:30:48.248449 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.248404 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/affca51d-9937-427c-9525-1a17670a9463-metrics-client-ca\") pod \"telemeter-client-7b57485f5f-89v49\" (UID: \"affca51d-9937-427c-9525-1a17670a9463\") " pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.248614 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.248458 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/affca51d-9937-427c-9525-1a17670a9463-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b57485f5f-89v49\" (UID: \"affca51d-9937-427c-9525-1a17670a9463\") " pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.248614 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.248496 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/affca51d-9937-427c-9525-1a17670a9463-serving-certs-ca-bundle\") pod \"telemeter-client-7b57485f5f-89v49\" (UID: \"affca51d-9937-427c-9525-1a17670a9463\") " pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.248614 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.248605 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/affca51d-9937-427c-9525-1a17670a9463-federate-client-tls\") pod \"telemeter-client-7b57485f5f-89v49\" (UID: \"affca51d-9937-427c-9525-1a17670a9463\") " pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.248772 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.248629 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/affca51d-9937-427c-9525-1a17670a9463-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b57485f5f-89v49\" (UID: \"affca51d-9937-427c-9525-1a17670a9463\") " pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.248772 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.248657 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/affca51d-9937-427c-9525-1a17670a9463-secret-telemeter-client\") pod \"telemeter-client-7b57485f5f-89v49\" (UID: \"affca51d-9937-427c-9525-1a17670a9463\") " pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.248772 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.248687 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shtzn\" (UniqueName: \"kubernetes.io/projected/affca51d-9937-427c-9525-1a17670a9463-kube-api-access-shtzn\") pod \"telemeter-client-7b57485f5f-89v49\" (UID: \"affca51d-9937-427c-9525-1a17670a9463\") " pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.248772 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.248713 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/affca51d-9937-427c-9525-1a17670a9463-telemeter-client-tls\") pod \"telemeter-client-7b57485f5f-89v49\" (UID: \"affca51d-9937-427c-9525-1a17670a9463\") " pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.293762 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.293725 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" event={"ID":"b2d57361-d50a-47a0-9f50-d308478d019a","Type":"ContainerStarted","Data":"d2a0c9ad2595e7f82dfbeed0854e2c6a0e7c7cfc4cb54833dd473c8f67c26aeb"} Jul 09 13:30:48.294208 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.293799 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" event={"ID":"b2d57361-d50a-47a0-9f50-d308478d019a","Type":"ContainerStarted","Data":"59fa2fb96920451c920a5b81e3d0003f74eac7afa47eb7dced90f649922bab55"} Jul 09 13:30:48.294208 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.293835 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" event={"ID":"b2d57361-d50a-47a0-9f50-d308478d019a","Type":"ContainerStarted","Data":"8368ec81e440b08d42de92ebd2259f80cb1e58c43b3e8d512e48699ef7c0d48a"} Jul 09 13:30:48.294944 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.294919 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" event={"ID":"43a7ad9e-a15f-46d4-8acb-791cd9a2300b","Type":"ContainerStarted","Data":"3fcae5bdee6eac226cba4971ec83cb7cb99d9a8afe6fc39e67478fd91ad15625"} Jul 09 13:30:48.297052 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.297001 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pq8wj" event={"ID":"edec525a-47f6-4c88-92c8-4e502b6f2739","Type":"ContainerStarted","Data":"1312e1bcd21456dd9cec1ad49942b2b9e7a3f09b441f1ad02a46a254377b4181"} Jul 09 13:30:48.297154 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.297067 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pq8wj" event={"ID":"edec525a-47f6-4c88-92c8-4e502b6f2739","Type":"ContainerStarted","Data":"16116296c78c5aded738c07c10066680a48b96c9d5a924b87b619fb5683aaeef"} Jul 09 13:30:48.329617 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.329563 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-pq8wj" podStartSLOduration=4.633631602 podStartE2EDuration="6.329547964s" totalImagesPullingTime="1.595911669s" totalInitContainerRuntime="100.004693ms" isStatefulPod=true podCreationTimestamp="2026-07-09 13:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:48.329137062 +0000 UTC m=+65.834439329" watchObservedRunningTime="2026-07-09 13:30:48.329547964 +0000 UTC m=+65.834850277" Jul 09 13:30:48.350094 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.350058 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/affca51d-9937-427c-9525-1a17670a9463-metrics-client-ca\") pod \"telemeter-client-7b57485f5f-89v49\" (UID: \"affca51d-9937-427c-9525-1a17670a9463\") " pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.350285 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.350108 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/affca51d-9937-427c-9525-1a17670a9463-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b57485f5f-89v49\" (UID: \"affca51d-9937-427c-9525-1a17670a9463\") " pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.350285 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.350166 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/affca51d-9937-427c-9525-1a17670a9463-serving-certs-ca-bundle\") pod \"telemeter-client-7b57485f5f-89v49\" (UID: \"affca51d-9937-427c-9525-1a17670a9463\") " pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.350404 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.350354 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2c41244e-abc4-4b1f-8c04-3d74077d0e6e-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-qblcv\" (UID: \"2c41244e-abc4-4b1f-8c04-3d74077d0e6e\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qblcv" Jul 09 13:30:48.350404 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.350399 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/affca51d-9937-427c-9525-1a17670a9463-federate-client-tls\") pod \"telemeter-client-7b57485f5f-89v49\" (UID: \"affca51d-9937-427c-9525-1a17670a9463\") " pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.350501 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.350417 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/affca51d-9937-427c-9525-1a17670a9463-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b57485f5f-89v49\" (UID: \"affca51d-9937-427c-9525-1a17670a9463\") " pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.350501 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.350437 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/affca51d-9937-427c-9525-1a17670a9463-secret-telemeter-client\") pod \"telemeter-client-7b57485f5f-89v49\" (UID: \"affca51d-9937-427c-9525-1a17670a9463\") " pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.350501 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.350459 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shtzn\" (UniqueName: \"kubernetes.io/projected/affca51d-9937-427c-9525-1a17670a9463-kube-api-access-shtzn\") pod \"telemeter-client-7b57485f5f-89v49\" (UID: \"affca51d-9937-427c-9525-1a17670a9463\") " pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.350501 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.350479 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/affca51d-9937-427c-9525-1a17670a9463-telemeter-client-tls\") pod \"telemeter-client-7b57485f5f-89v49\" (UID: \"affca51d-9937-427c-9525-1a17670a9463\") " pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.351720 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.350951 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/affca51d-9937-427c-9525-1a17670a9463-metrics-client-ca\") pod \"telemeter-client-7b57485f5f-89v49\" (UID: \"affca51d-9937-427c-9525-1a17670a9463\") " pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.351720 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.351152 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/affca51d-9937-427c-9525-1a17670a9463-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b57485f5f-89v49\" (UID: \"affca51d-9937-427c-9525-1a17670a9463\") " pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.351720 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.351211 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/affca51d-9937-427c-9525-1a17670a9463-serving-certs-ca-bundle\") pod \"telemeter-client-7b57485f5f-89v49\" (UID: \"affca51d-9937-427c-9525-1a17670a9463\") " pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.353891 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.353865 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2c41244e-abc4-4b1f-8c04-3d74077d0e6e-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-qblcv\" (UID: \"2c41244e-abc4-4b1f-8c04-3d74077d0e6e\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qblcv" Jul 09 13:30:48.354047 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.353910 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/affca51d-9937-427c-9525-1a17670a9463-secret-telemeter-client\") pod \"telemeter-client-7b57485f5f-89v49\" (UID: \"affca51d-9937-427c-9525-1a17670a9463\") " pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.354047 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.353989 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/affca51d-9937-427c-9525-1a17670a9463-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b57485f5f-89v49\" (UID: \"affca51d-9937-427c-9525-1a17670a9463\") " pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.354506 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.354481 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/affca51d-9937-427c-9525-1a17670a9463-federate-client-tls\") pod \"telemeter-client-7b57485f5f-89v49\" (UID: \"affca51d-9937-427c-9525-1a17670a9463\") " pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.354568 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.354486 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/affca51d-9937-427c-9525-1a17670a9463-telemeter-client-tls\") pod \"telemeter-client-7b57485f5f-89v49\" (UID: \"affca51d-9937-427c-9525-1a17670a9463\") " pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.369394 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.369358 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shtzn\" (UniqueName: \"kubernetes.io/projected/affca51d-9937-427c-9525-1a17670a9463-kube-api-access-shtzn\") pod \"telemeter-client-7b57485f5f-89v49\" (UID: \"affca51d-9937-427c-9525-1a17670a9463\") " pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.525676 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.525368 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" Jul 09 13:30:48.627389 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.627352 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qblcv" Jul 09 13:30:48.641503 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.640871 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56c6f87bc5-824bh"] Jul 09 13:30:48.671072 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.670581 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58d464df88-gfglq"] Jul 09 13:30:48.675302 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.675280 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:30:48.688769 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.688532 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58d464df88-gfglq"] Jul 09 13:30:48.706158 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.706057 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7b57485f5f-89v49"] Jul 09 13:30:48.754228 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.754194 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc9sj\" (UniqueName: \"kubernetes.io/projected/92ffc297-fe13-4928-8ea7-34e31c8c3884-kube-api-access-kc9sj\") pod \"console-58d464df88-gfglq\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:30:48.754392 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.754288 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-metrics-certs\") pod \"network-metrics-daemon-5zl2x\" (UID: \"fbe44bc5-d23a-4535-b1d8-74ec52d0ce47\") " pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:30:48.754392 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.754320 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/92ffc297-fe13-4928-8ea7-34e31c8c3884-console-oauth-config\") pod \"console-58d464df88-gfglq\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:30:48.754392 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.754365 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-original-pull-secret\") pod \"global-pull-secret-syncer-65fnv\" (UID: \"fb20e4df-4233-4fb6-a1f7-e58a789e3d56\") " pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:30:48.754549 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.754424 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/92ffc297-fe13-4928-8ea7-34e31c8c3884-console-config\") pod \"console-58d464df88-gfglq\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:30:48.754549 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.754471 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/92ffc297-fe13-4928-8ea7-34e31c8c3884-oauth-serving-cert\") pod \"console-58d464df88-gfglq\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:30:48.754549 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.754501 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92ffc297-fe13-4928-8ea7-34e31c8c3884-trusted-ca-bundle\") pod \"console-58d464df88-gfglq\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:30:48.754549 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.754540 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/92ffc297-fe13-4928-8ea7-34e31c8c3884-console-serving-cert\") pod \"console-58d464df88-gfglq\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:30:48.754733 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.754574 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/92ffc297-fe13-4928-8ea7-34e31c8c3884-service-ca\") pod \"console-58d464df88-gfglq\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:30:48.757639 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.757604 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 13:30:48.757903 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.757884 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 13:30:48.768233 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.768200 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbe44bc5-d23a-4535-b1d8-74ec52d0ce47-metrics-certs\") pod \"network-metrics-daemon-5zl2x\" (UID: \"fbe44bc5-d23a-4535-b1d8-74ec52d0ce47\") " pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:30:48.770767 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.770703 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb20e4df-4233-4fb6-a1f7-e58a789e3d56-original-pull-secret\") pod \"global-pull-secret-syncer-65fnv\" (UID: \"fb20e4df-4233-4fb6-a1f7-e58a789e3d56\") " pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:30:48.772741 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.772594 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-65fnv" Jul 09 13:30:48.845355 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.845322 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-qblcv"] Jul 09 13:30:48.855812 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.855772 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/92ffc297-fe13-4928-8ea7-34e31c8c3884-console-config\") pod \"console-58d464df88-gfglq\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:30:48.855979 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.855826 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/92ffc297-fe13-4928-8ea7-34e31c8c3884-oauth-serving-cert\") pod \"console-58d464df88-gfglq\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:30:48.855979 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.855855 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92ffc297-fe13-4928-8ea7-34e31c8c3884-trusted-ca-bundle\") pod \"console-58d464df88-gfglq\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:30:48.855979 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.855884 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/92ffc297-fe13-4928-8ea7-34e31c8c3884-console-serving-cert\") pod \"console-58d464df88-gfglq\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:30:48.855979 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.855913 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/92ffc297-fe13-4928-8ea7-34e31c8c3884-service-ca\") pod \"console-58d464df88-gfglq\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:30:48.855979 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.855962 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc9sj\" (UniqueName: \"kubernetes.io/projected/92ffc297-fe13-4928-8ea7-34e31c8c3884-kube-api-access-kc9sj\") pod \"console-58d464df88-gfglq\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:30:48.856244 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.855997 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjqpc\" (UniqueName: \"kubernetes.io/projected/9e8320e4-2106-4edf-b225-d65a4b3eae55-kube-api-access-wjqpc\") pod \"network-check-target-h6qqn\" (UID: \"9e8320e4-2106-4edf-b225-d65a4b3eae55\") " pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:30:48.856244 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.856076 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/92ffc297-fe13-4928-8ea7-34e31c8c3884-console-oauth-config\") pod \"console-58d464df88-gfglq\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:30:48.857200 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.857139 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/92ffc297-fe13-4928-8ea7-34e31c8c3884-service-ca\") pod \"console-58d464df88-gfglq\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:30:48.857200 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.857163 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/92ffc297-fe13-4928-8ea7-34e31c8c3884-console-config\") pod \"console-58d464df88-gfglq\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:30:48.857789 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.857764 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/92ffc297-fe13-4928-8ea7-34e31c8c3884-oauth-serving-cert\") pod \"console-58d464df88-gfglq\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:30:48.858200 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.858164 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92ffc297-fe13-4928-8ea7-34e31c8c3884-trusted-ca-bundle\") pod \"console-58d464df88-gfglq\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:30:48.859455 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.859405 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/92ffc297-fe13-4928-8ea7-34e31c8c3884-console-oauth-config\") pod \"console-58d464df88-gfglq\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:30:48.860176 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.860155 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/92ffc297-fe13-4928-8ea7-34e31c8c3884-console-serving-cert\") pod \"console-58d464df88-gfglq\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:30:48.861080 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.861055 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 13:30:48.870172 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.870141 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 13:30:48.874391 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.874302 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc9sj\" (UniqueName: \"kubernetes.io/projected/92ffc297-fe13-4928-8ea7-34e31c8c3884-kube-api-access-kc9sj\") pod \"console-58d464df88-gfglq\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:30:48.880332 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.880272 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjqpc\" (UniqueName: \"kubernetes.io/projected/9e8320e4-2106-4edf-b225-d65a4b3eae55-kube-api-access-wjqpc\") pod \"network-check-target-h6qqn\" (UID: \"9e8320e4-2106-4edf-b225-d65a4b3eae55\") " pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:30:48.956707 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.956613 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wmhp6\"" Jul 09 13:30:48.962845 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.962816 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5zl2x" Jul 09 13:30:48.997280 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:48.995328 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:30:49.066285 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.066205 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xg9bl\"" Jul 09 13:30:49.074901 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.074869 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:30:49.307329 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:30:49.307291 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaffca51d_9937_427c_9525_1a17670a9463.slice/crio-24d76fee71c1d3ed20b4b3c55ce26024c9a453881b5fc213c195a72b141214fa WatchSource:0}: Error finding container 24d76fee71c1d3ed20b4b3c55ce26024c9a453881b5fc213c195a72b141214fa: Status 404 returned error can't find the container with id 24d76fee71c1d3ed20b4b3c55ce26024c9a453881b5fc213c195a72b141214fa Jul 09 13:30:49.308852 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:30:49.308825 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c41244e_abc4_4b1f_8c04_3d74077d0e6e.slice/crio-1d8fd89e858ce2a1243040af5f7c8bc3b71e2dc73377cdac711d9d65312212c3 WatchSource:0}: Error finding container 1d8fd89e858ce2a1243040af5f7c8bc3b71e2dc73377cdac711d9d65312212c3: Status 404 returned error can't find the container with id 1d8fd89e858ce2a1243040af5f7c8bc3b71e2dc73377cdac711d9d65312212c3 Jul 09 13:30:49.315078 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.313236 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 13:30:49.317223 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.317111 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.320456 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.320153 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 09 13:30:49.320456 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.320250 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 09 13:30:49.320456 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.320258 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 09 13:30:49.320456 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.320273 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ht5xk\"" Jul 09 13:30:49.320456 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.320246 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 09 13:30:49.320456 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.320389 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 09 13:30:49.320807 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.320653 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 09 13:30:49.323630 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.321185 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 09 13:30:49.323630 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.321605 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 09 13:30:49.323630 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.321826 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 09 13:30:49.323630 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.322062 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 09 13:30:49.323630 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.323197 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 09 13:30:49.323630 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.323295 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9octbmd59c86i\"" Jul 09 13:30:49.323630 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.323471 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 09 13:30:49.325747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.325563 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 09 13:30:49.328348 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.328316 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 09 13:30:49.338764 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.336669 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 13:30:49.360870 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.360820 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.361056 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.360873 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.361056 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.360907 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.361056 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.360952 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.361056 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.360980 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.361056 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.361054 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.361328 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.361083 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8ttr\" (UniqueName: \"kubernetes.io/projected/413c5fe9-a023-4ec7-9e45-6141b411d1a1-kube-api-access-c8ttr\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.361328 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.361117 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/413c5fe9-a023-4ec7-9e45-6141b411d1a1-config-out\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.361328 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.361161 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.361328 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.361186 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.361328 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.361214 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.361328 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.361236 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.361328 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.361289 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.361328 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.361311 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-web-config\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.361717 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.361366 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.361717 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.361397 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.361717 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.361421 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.361717 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.361509 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/413c5fe9-a023-4ec7-9e45-6141b411d1a1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.361717 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.361531 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.361717 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.361586 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-config\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.463052 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.462797 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.463052 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.462848 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.463052 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.462875 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.463052 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.462932 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/413c5fe9-a023-4ec7-9e45-6141b411d1a1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.463052 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.462954 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.463052 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.462992 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-config\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.464099 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.463035 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.464099 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.463535 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.464099 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.463565 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.464099 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.463601 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.464099 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.463630 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.464099 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.463661 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.464099 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.463668 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.464099 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.463695 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8ttr\" (UniqueName: \"kubernetes.io/projected/413c5fe9-a023-4ec7-9e45-6141b411d1a1-kube-api-access-c8ttr\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.464099 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.463735 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/413c5fe9-a023-4ec7-9e45-6141b411d1a1-config-out\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.464099 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.463770 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.464099 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.463794 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.464099 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.463826 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.464099 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.463852 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.464099 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.463900 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.464099 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.463927 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-web-config\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.464849 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.464344 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.464849 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.464593 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.465745 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.465249 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.466323 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.466010 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.467477 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.467111 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.467477 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.467247 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.468905 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.468543 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.469431 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.469395 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.471568 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.471539 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.471698 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.471678 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.472087 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.472009 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/413c5fe9-a023-4ec7-9e45-6141b411d1a1-config-out\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.472269 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.472230 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.472426 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.472403 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.472502 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.472476 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.472703 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.472682 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-web-config\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.473082 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.473059 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/413c5fe9-a023-4ec7-9e45-6141b411d1a1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.473204 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.473184 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-config\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.473847 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.473822 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.478038 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.477986 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8ttr\" (UniqueName: \"kubernetes.io/projected/413c5fe9-a023-4ec7-9e45-6141b411d1a1-kube-api-access-c8ttr\") pod \"prometheus-k8s-0\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:49.643213 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:49.643171 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:30:50.188296 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:50.188254 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 13:30:50.221246 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:50.221071 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-65fnv"] Jul 09 13:30:50.229200 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:30:50.229159 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb20e4df_4233_4fb6_a1f7_e58a789e3d56.slice/crio-6f3c8588461eaada9dc662bca83aa134d72dcf11f2b64b98f7cf352e74b2039e WatchSource:0}: Error finding container 6f3c8588461eaada9dc662bca83aa134d72dcf11f2b64b98f7cf352e74b2039e: Status 404 returned error can't find the container with id 6f3c8588461eaada9dc662bca83aa134d72dcf11f2b64b98f7cf352e74b2039e Jul 09 13:30:50.314149 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:50.314049 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-65fnv" event={"ID":"fb20e4df-4233-4fb6-a1f7-e58a789e3d56","Type":"ContainerStarted","Data":"6f3c8588461eaada9dc662bca83aa134d72dcf11f2b64b98f7cf352e74b2039e"} Jul 09 13:30:50.317790 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:50.317660 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"413c5fe9-a023-4ec7-9e45-6141b411d1a1","Type":"ContainerStarted","Data":"d9e61cfad5c0de7d250ab81073b854c3ea3b1de4c2c26c38d337e601994da271"} Jul 09 13:30:50.334468 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:50.334296 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" event={"ID":"b2d57361-d50a-47a0-9f50-d308478d019a","Type":"ContainerStarted","Data":"a8e0d851578a4b480dc87fda882d8e212c58bbf4591b20d1f45ddf08d33dc574"} Jul 09 13:30:50.334468 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:50.334390 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" event={"ID":"b2d57361-d50a-47a0-9f50-d308478d019a","Type":"ContainerStarted","Data":"2f5e2ae32e4128f60fac1f5ba212095daffbbe0aadbd05c50f8ec1c1fb727865"} Jul 09 13:30:50.355614 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:50.355199 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" event={"ID":"affca51d-9937-427c-9525-1a17670a9463","Type":"ContainerStarted","Data":"24d76fee71c1d3ed20b4b3c55ce26024c9a453881b5fc213c195a72b141214fa"} Jul 09 13:30:50.357416 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:50.357330 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qblcv" event={"ID":"2c41244e-abc4-4b1f-8c04-3d74077d0e6e","Type":"ContainerStarted","Data":"1d8fd89e858ce2a1243040af5f7c8bc3b71e2dc73377cdac711d9d65312212c3"} Jul 09 13:30:50.374947 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:50.374908 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b87b16cf-44bd-4ef7-99b3-cf305030612c","Type":"ContainerStarted","Data":"b1fe1e73eacc61c6471af3e47703023d2e9b59240bd46d56c6c1de323ea37a59"} Jul 09 13:30:50.375132 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:50.374997 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b87b16cf-44bd-4ef7-99b3-cf305030612c","Type":"ContainerStarted","Data":"9b2e2c6afdc5e29cacfca2837e78db7ed1d914e5576bc6b5023a1049d2de7b61"} Jul 09 13:30:50.377329 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:50.377292 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" event={"ID":"43a7ad9e-a15f-46d4-8acb-791cd9a2300b","Type":"ContainerStarted","Data":"3e487ee6efa1de9726a710037199708580ed754d5c2995e27b30f3f4877cca8f"} Jul 09 13:30:50.384344 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:50.384316 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-h6qqn"] Jul 09 13:30:50.387947 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:50.387916 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58d464df88-gfglq"] Jul 09 13:30:50.390939 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:50.390912 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5zl2x"] Jul 09 13:30:50.400278 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:50.400172 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" podStartSLOduration=1.495322949 podStartE2EDuration="3.400150891s" totalImagesPullingTime="1.904827942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:30:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:50.399787767 +0000 UTC m=+67.905090034" watchObservedRunningTime="2026-07-09 13:30:50.400150891 +0000 UTC m=+67.905453160" Jul 09 13:30:50.402585 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:30:50.402500 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e8320e4_2106_4edf_b225_d65a4b3eae55.slice/crio-79478157cc392332a0b957005e86038e86a920a840b7c51bcce230700f6854f8 WatchSource:0}: Error finding container 79478157cc392332a0b957005e86038e86a920a840b7c51bcce230700f6854f8: Status 404 returned error can't find the container with id 79478157cc392332a0b957005e86038e86a920a840b7c51bcce230700f6854f8 Jul 09 13:30:50.519720 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:50.519686 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:30:51.389528 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:51.388953 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" event={"ID":"b2d57361-d50a-47a0-9f50-d308478d019a","Type":"ContainerStarted","Data":"51d58fa442ffc03d5f45f73fd70b60463c0f4a17911ce62b58ad1b0f44e88670"} Jul 09 13:30:51.389528 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:51.389521 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:51.394259 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:51.394216 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b87b16cf-44bd-4ef7-99b3-cf305030612c","Type":"ContainerStarted","Data":"6af9f0d84987ed7781b4e0ba6fd8834c338a8c52c48b6ce98e30a65a203164d5"} Jul 09 13:30:51.394416 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:51.394299 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b87b16cf-44bd-4ef7-99b3-cf305030612c","Type":"ContainerStarted","Data":"004f34184e4b3eaaff06d5fc775fdbd7d2bbebdbaacae92821e3225297bd2e6f"} Jul 09 13:30:51.394416 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:51.394343 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b87b16cf-44bd-4ef7-99b3-cf305030612c","Type":"ContainerStarted","Data":"f3d316af84c32c32fb64fb24ecfab1f04fb2e2eb5766de3456573f8714bea904"} Jul 09 13:30:51.394416 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:51.394391 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b87b16cf-44bd-4ef7-99b3-cf305030612c","Type":"ContainerStarted","Data":"e4dfda9ab8923dd0ee2fc3f4840c487a3efadfb43e83b004681fc768f29717d8"} Jul 09 13:30:51.396544 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:51.396349 2573 generic.go:356] "Generic (PLEG): container finished" podID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerID="9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c" exitCode=0 Jul 09 13:30:51.396544 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:51.396440 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"413c5fe9-a023-4ec7-9e45-6141b411d1a1","Type":"ContainerDied","Data":"9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c"} Jul 09 13:30:51.398126 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:51.397948 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5zl2x" event={"ID":"fbe44bc5-d23a-4535-b1d8-74ec52d0ce47","Type":"ContainerStarted","Data":"918cfe72cf6ef5870fe8ca89465326be75a018b661a17aaf57e26ddcdd360b41"} Jul 09 13:30:51.399697 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:51.399648 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-h6qqn" event={"ID":"9e8320e4-2106-4edf-b225-d65a4b3eae55","Type":"ContainerStarted","Data":"79478157cc392332a0b957005e86038e86a920a840b7c51bcce230700f6854f8"} Jul 09 13:30:51.401893 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:51.401819 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58d464df88-gfglq" event={"ID":"92ffc297-fe13-4928-8ea7-34e31c8c3884","Type":"ContainerStarted","Data":"9b279564de5a22f06d14b869af9b3b77494eb7016ce47bb8db46d0d4459da0e1"} Jul 09 13:30:51.401893 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:51.401867 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58d464df88-gfglq" event={"ID":"92ffc297-fe13-4928-8ea7-34e31c8c3884","Type":"ContainerStarted","Data":"01bc63cb8b8e4155f32e555e2274a8bc8f59b1c0abb52c72cc219b9e2f9c6714"} Jul 09 13:30:51.416327 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:51.416261 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" podStartSLOduration=2.505427049 podStartE2EDuration="6.416239127s" totalImagesPullingTime="3.910812078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:30:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:51.414770046 +0000 UTC m=+68.920072354" watchObservedRunningTime="2026-07-09 13:30:51.416239127 +0000 UTC m=+68.921541375" Jul 09 13:30:51.467449 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:51.467395 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58d464df88-gfglq" podStartSLOduration=3.46737725 podStartE2EDuration="3.46737725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:30:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:51.466360833 +0000 UTC m=+68.971663102" watchObservedRunningTime="2026-07-09 13:30:51.46737725 +0000 UTC m=+68.972679496" Jul 09 13:30:51.498610 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:51.498508 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.56595753 podStartE2EDuration="7.498482243s" totalImagesPullingTime="3.912847294s" totalInitContainerRuntime="19.677419ms" isStatefulPod=false podCreationTimestamp="2026-07-09 13:30:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:51.496849602 +0000 UTC m=+69.002151891" watchObservedRunningTime="2026-07-09 13:30:51.498482243 +0000 UTC m=+69.003784518" Jul 09 13:30:52.418008 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:52.417975 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6d5688b76d-th79t" Jul 09 13:30:52.991399 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:30:52.991342 2573 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaffca51d_9937_427c_9525_1a17670a9463.slice/crio-c5fd56fa28ff8235e195642c4af789a7e3cb61017a9282c514d26cfc4cd96e93.scope\": RecentStats: unable to find data in memory cache]" Jul 09 13:30:53.419529 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:53.419450 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qblcv" event={"ID":"2c41244e-abc4-4b1f-8c04-3d74077d0e6e","Type":"ContainerStarted","Data":"426dc78de782e86ad1429055ae348e03dd8e29e2a8a8017e728d7cd559d68d3a"} Jul 09 13:30:53.421550 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:53.420280 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qblcv" Jul 09 13:30:53.423351 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:53.423319 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" event={"ID":"affca51d-9937-427c-9525-1a17670a9463","Type":"ContainerStarted","Data":"261f071a911825a37dc6987366889cc8a2a42f8311787e5910e230183845a29a"} Jul 09 13:30:53.423454 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:53.423382 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" event={"ID":"affca51d-9937-427c-9525-1a17670a9463","Type":"ContainerStarted","Data":"2fbe90763b369e874488be87f022cc4f1dfc680f38aac6335e21e9521038e155"} Jul 09 13:30:53.423454 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:53.423415 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" event={"ID":"affca51d-9937-427c-9525-1a17670a9463","Type":"ContainerStarted","Data":"c5fd56fa28ff8235e195642c4af789a7e3cb61017a9282c514d26cfc4cd96e93"} Jul 09 13:30:53.427429 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:53.427382 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qblcv" Jul 09 13:30:53.429064 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:53.428955 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5zl2x" event={"ID":"fbe44bc5-d23a-4535-b1d8-74ec52d0ce47","Type":"ContainerStarted","Data":"32c770684925ea053985478206cd8f6ea42760e85bd92269751b984bf5c49935"} Jul 09 13:30:53.429064 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:53.429004 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5zl2x" event={"ID":"fbe44bc5-d23a-4535-b1d8-74ec52d0ce47","Type":"ContainerStarted","Data":"cd5df735059908bc2a05e06e587851d9d5511a642e2ed697f65bce8895922cb9"} Jul 09 13:30:53.441012 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:53.440948 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qblcv" podStartSLOduration=2.881884559 podStartE2EDuration="6.440921209s" totalImagesPullingTime="3.55903665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:30:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:53.439547345 +0000 UTC m=+70.944849618" watchObservedRunningTime="2026-07-09 13:30:53.440921209 +0000 UTC m=+70.946223478" Jul 09 13:30:53.487036 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:53.485640 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7b57485f5f-89v49" podStartSLOduration=1.911577327 podStartE2EDuration="5.485615579s" totalImagesPullingTime="3.574038252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:30:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:53.484434256 +0000 UTC m=+70.989736535" watchObservedRunningTime="2026-07-09 13:30:53.485615579 +0000 UTC m=+70.990917849" Jul 09 13:30:53.507839 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:53.507483 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5zl2x" podStartSLOduration=67.999369873 podStartE2EDuration="1m10.507455167s" totalImagesPullingTime="2.508085294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:29:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:53.50497529 +0000 UTC m=+71.010277559" watchObservedRunningTime="2026-07-09 13:30:53.507455167 +0000 UTC m=+71.012757436" Jul 09 13:30:54.560424 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.560387 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58d464df88-gfglq"] Jul 09 13:30:54.616474 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.616433 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f7f8d56cf-4nrfj"] Jul 09 13:30:54.620379 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.619797 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:30:54.636859 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.636821 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f7f8d56cf-4nrfj"] Jul 09 13:30:54.747361 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.747317 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/56a9cac0-8d97-4b72-b8c0-b3adf507d495-console-config\") pod \"console-5f7f8d56cf-4nrfj\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:30:54.747567 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.747371 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/56a9cac0-8d97-4b72-b8c0-b3adf507d495-oauth-serving-cert\") pod \"console-5f7f8d56cf-4nrfj\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:30:54.747567 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.747405 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/56a9cac0-8d97-4b72-b8c0-b3adf507d495-console-serving-cert\") pod \"console-5f7f8d56cf-4nrfj\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:30:54.747567 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.747490 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56a9cac0-8d97-4b72-b8c0-b3adf507d495-trusted-ca-bundle\") pod \"console-5f7f8d56cf-4nrfj\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:30:54.747567 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.747530 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74ztf\" (UniqueName: \"kubernetes.io/projected/56a9cac0-8d97-4b72-b8c0-b3adf507d495-kube-api-access-74ztf\") pod \"console-5f7f8d56cf-4nrfj\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:30:54.747567 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.747562 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/56a9cac0-8d97-4b72-b8c0-b3adf507d495-console-oauth-config\") pod \"console-5f7f8d56cf-4nrfj\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:30:54.747788 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.747586 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/56a9cac0-8d97-4b72-b8c0-b3adf507d495-service-ca\") pod \"console-5f7f8d56cf-4nrfj\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:30:54.849177 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.849103 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56a9cac0-8d97-4b72-b8c0-b3adf507d495-trusted-ca-bundle\") pod \"console-5f7f8d56cf-4nrfj\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:30:54.849177 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.849155 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74ztf\" (UniqueName: \"kubernetes.io/projected/56a9cac0-8d97-4b72-b8c0-b3adf507d495-kube-api-access-74ztf\") pod \"console-5f7f8d56cf-4nrfj\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:30:54.849373 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.849192 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/56a9cac0-8d97-4b72-b8c0-b3adf507d495-console-oauth-config\") pod \"console-5f7f8d56cf-4nrfj\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:30:54.849373 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.849214 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/56a9cac0-8d97-4b72-b8c0-b3adf507d495-service-ca\") pod \"console-5f7f8d56cf-4nrfj\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:30:54.849373 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.849289 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/56a9cac0-8d97-4b72-b8c0-b3adf507d495-console-config\") pod \"console-5f7f8d56cf-4nrfj\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:30:54.849373 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.849321 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/56a9cac0-8d97-4b72-b8c0-b3adf507d495-oauth-serving-cert\") pod \"console-5f7f8d56cf-4nrfj\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:30:54.849373 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.849352 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/56a9cac0-8d97-4b72-b8c0-b3adf507d495-console-serving-cert\") pod \"console-5f7f8d56cf-4nrfj\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:30:54.850344 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.849987 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/56a9cac0-8d97-4b72-b8c0-b3adf507d495-service-ca\") pod \"console-5f7f8d56cf-4nrfj\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:30:54.850344 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.850059 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56a9cac0-8d97-4b72-b8c0-b3adf507d495-trusted-ca-bundle\") pod \"console-5f7f8d56cf-4nrfj\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:30:54.850344 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.850307 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/56a9cac0-8d97-4b72-b8c0-b3adf507d495-console-config\") pod \"console-5f7f8d56cf-4nrfj\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:30:54.850620 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.850602 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/56a9cac0-8d97-4b72-b8c0-b3adf507d495-oauth-serving-cert\") pod \"console-5f7f8d56cf-4nrfj\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:30:54.852869 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.852834 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/56a9cac0-8d97-4b72-b8c0-b3adf507d495-console-serving-cert\") pod \"console-5f7f8d56cf-4nrfj\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:30:54.853472 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.853425 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/56a9cac0-8d97-4b72-b8c0-b3adf507d495-console-oauth-config\") pod \"console-5f7f8d56cf-4nrfj\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:30:54.858745 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.858706 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74ztf\" (UniqueName: \"kubernetes.io/projected/56a9cac0-8d97-4b72-b8c0-b3adf507d495-kube-api-access-74ztf\") pod \"console-5f7f8d56cf-4nrfj\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:30:54.936293 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:54.936252 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:30:57.453331 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:57.453245 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-65fnv" event={"ID":"fb20e4df-4233-4fb6-a1f7-e58a789e3d56","Type":"ContainerStarted","Data":"658f7b7a07d4ed842307cb8c61534542657576726b761f4c2ed2893d3bfcfb54"} Jul 09 13:30:57.455854 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:57.455801 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"413c5fe9-a023-4ec7-9e45-6141b411d1a1","Type":"ContainerStarted","Data":"b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c"} Jul 09 13:30:57.458154 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:57.458117 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-h6qqn" event={"ID":"9e8320e4-2106-4edf-b225-d65a4b3eae55","Type":"ContainerStarted","Data":"ceb65d8c488fc3f322c7a35eac35cc423d47de77f75fc18861db792e1de5015a"} Jul 09 13:30:57.458320 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:57.458213 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:30:57.463507 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:57.463427 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f7f8d56cf-4nrfj"] Jul 09 13:30:57.473542 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:57.473421 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-65fnv" podStartSLOduration=67.402477806 podStartE2EDuration="1m14.473398698s" totalImagesPullingTime="7.070920892s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:29:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:57.471284327 +0000 UTC m=+74.976586595" watchObservedRunningTime="2026-07-09 13:30:57.473398698 +0000 UTC m=+74.978700949" Jul 09 13:30:57.493396 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:57.493339 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-h6qqn" podStartSLOduration=67.603691031 podStartE2EDuration="1m14.493314312s" totalImagesPullingTime="6.889623281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:29:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:57.490682677 +0000 UTC m=+74.995984948" watchObservedRunningTime="2026-07-09 13:30:57.493314312 +0000 UTC m=+74.998616582" Jul 09 13:30:58.466395 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:58.466352 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"413c5fe9-a023-4ec7-9e45-6141b411d1a1","Type":"ContainerStarted","Data":"755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967"} Jul 09 13:30:58.466889 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:58.466473 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"413c5fe9-a023-4ec7-9e45-6141b411d1a1","Type":"ContainerStarted","Data":"d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae"} Jul 09 13:30:58.466889 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:58.466520 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"413c5fe9-a023-4ec7-9e45-6141b411d1a1","Type":"ContainerStarted","Data":"fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c"} Jul 09 13:30:58.466889 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:58.466565 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"413c5fe9-a023-4ec7-9e45-6141b411d1a1","Type":"ContainerStarted","Data":"abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784"} Jul 09 13:30:58.466889 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:58.466601 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"413c5fe9-a023-4ec7-9e45-6141b411d1a1","Type":"ContainerStarted","Data":"678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3"} Jul 09 13:30:58.467961 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:58.467932 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f7f8d56cf-4nrfj" event={"ID":"56a9cac0-8d97-4b72-b8c0-b3adf507d495","Type":"ContainerStarted","Data":"a1f81ebf2e8121b86409c8313e921cbbb5f8864a513c5fe5ec261b37e7fea66a"} Jul 09 13:30:58.468111 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:58.467975 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f7f8d56cf-4nrfj" event={"ID":"56a9cac0-8d97-4b72-b8c0-b3adf507d495","Type":"ContainerStarted","Data":"842ec12bb2518dcdab607b8a40da0d4a2351645044c1f00ed8f9cf84fac3b5e6"} Jul 09 13:30:58.500515 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:58.500459 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.564324597 podStartE2EDuration="9.500439958s" totalImagesPullingTime="5.909621264s" totalInitContainerRuntime="26.494097ms" isStatefulPod=false podCreationTimestamp="2026-07-09 13:30:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:58.497487779 +0000 UTC m=+76.002790065" watchObservedRunningTime="2026-07-09 13:30:58.500439958 +0000 UTC m=+76.005742238" Jul 09 13:30:58.518408 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:58.518358 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f7f8d56cf-4nrfj" podStartSLOduration=4.518340212 podStartE2EDuration="4.518340212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:30:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:30:58.517505096 +0000 UTC m=+76.022807356" watchObservedRunningTime="2026-07-09 13:30:58.518340212 +0000 UTC m=+76.023642481" Jul 09 13:30:58.995985 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:58.995936 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:30:59.643788 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:30:59.643744 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:31:04.936917 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:04.936779 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:31:04.936917 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:04.936840 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:31:04.950428 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:04.950399 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:31:05.505081 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:05.505048 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:31:05.559346 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:05.559309 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6578f4b68b-pwb9x"] Jul 09 13:31:07.756990 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:07.756950 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:31:07.757502 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:07.757068 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:31:13.692791 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:13.692742 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-56c6f87bc5-824bh" podUID="ff1595e1-86db-4f53-a118-dff9aa8c51cc" containerName="console" containerID="cri-o://cc7c12dfa092231fadcb76d9bd69930dce9c7f63b11fb3e7919344777c4408b7" gracePeriod=15 Jul 09 13:31:13.930975 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:13.930951 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:31:14.029276 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.029180 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff1595e1-86db-4f53-a118-dff9aa8c51cc-trusted-ca-bundle\") pod \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " Jul 09 13:31:14.029276 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.029243 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff1595e1-86db-4f53-a118-dff9aa8c51cc-console-serving-cert\") pod \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " Jul 09 13:31:14.029276 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.029265 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff1595e1-86db-4f53-a118-dff9aa8c51cc-oauth-serving-cert\") pod \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " Jul 09 13:31:14.029550 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.029315 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9jkg\" (UniqueName: \"kubernetes.io/projected/ff1595e1-86db-4f53-a118-dff9aa8c51cc-kube-api-access-z9jkg\") pod \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " Jul 09 13:31:14.029550 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.029395 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff1595e1-86db-4f53-a118-dff9aa8c51cc-service-ca\") pod \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " Jul 09 13:31:14.029550 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.029420 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff1595e1-86db-4f53-a118-dff9aa8c51cc-console-config\") pod \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " Jul 09 13:31:14.029550 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.029440 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff1595e1-86db-4f53-a118-dff9aa8c51cc-console-oauth-config\") pod \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\" (UID: \"ff1595e1-86db-4f53-a118-dff9aa8c51cc\") " Jul 09 13:31:14.029768 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.029726 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff1595e1-86db-4f53-a118-dff9aa8c51cc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ff1595e1-86db-4f53-a118-dff9aa8c51cc" (UID: "ff1595e1-86db-4f53-a118-dff9aa8c51cc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:31:14.029824 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.029785 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff1595e1-86db-4f53-a118-dff9aa8c51cc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ff1595e1-86db-4f53-a118-dff9aa8c51cc" (UID: "ff1595e1-86db-4f53-a118-dff9aa8c51cc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:31:14.029937 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.029912 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff1595e1-86db-4f53-a118-dff9aa8c51cc-console-config" (OuterVolumeSpecName: "console-config") pod "ff1595e1-86db-4f53-a118-dff9aa8c51cc" (UID: "ff1595e1-86db-4f53-a118-dff9aa8c51cc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:31:14.029991 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.029937 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff1595e1-86db-4f53-a118-dff9aa8c51cc-service-ca" (OuterVolumeSpecName: "service-ca") pod "ff1595e1-86db-4f53-a118-dff9aa8c51cc" (UID: "ff1595e1-86db-4f53-a118-dff9aa8c51cc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:31:14.031712 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.031685 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff1595e1-86db-4f53-a118-dff9aa8c51cc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ff1595e1-86db-4f53-a118-dff9aa8c51cc" (UID: "ff1595e1-86db-4f53-a118-dff9aa8c51cc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:31:14.031824 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.031702 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff1595e1-86db-4f53-a118-dff9aa8c51cc-kube-api-access-z9jkg" (OuterVolumeSpecName: "kube-api-access-z9jkg") pod "ff1595e1-86db-4f53-a118-dff9aa8c51cc" (UID: "ff1595e1-86db-4f53-a118-dff9aa8c51cc"). InnerVolumeSpecName "kube-api-access-z9jkg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 13:31:14.031824 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.031758 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff1595e1-86db-4f53-a118-dff9aa8c51cc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ff1595e1-86db-4f53-a118-dff9aa8c51cc" (UID: "ff1595e1-86db-4f53-a118-dff9aa8c51cc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:31:14.131088 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.131053 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff1595e1-86db-4f53-a118-dff9aa8c51cc-console-config\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:31:14.131088 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.131083 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff1595e1-86db-4f53-a118-dff9aa8c51cc-console-oauth-config\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:31:14.131088 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.131094 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff1595e1-86db-4f53-a118-dff9aa8c51cc-trusted-ca-bundle\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:31:14.131305 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.131103 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff1595e1-86db-4f53-a118-dff9aa8c51cc-console-serving-cert\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:31:14.131305 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.131113 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff1595e1-86db-4f53-a118-dff9aa8c51cc-oauth-serving-cert\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:31:14.131305 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.131123 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9jkg\" (UniqueName: \"kubernetes.io/projected/ff1595e1-86db-4f53-a118-dff9aa8c51cc-kube-api-access-z9jkg\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:31:14.131305 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.131131 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff1595e1-86db-4f53-a118-dff9aa8c51cc-service-ca\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:31:14.542705 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.542665 2573 generic.go:356] "Generic (PLEG): container finished" podID="ff1595e1-86db-4f53-a118-dff9aa8c51cc" containerID="cc7c12dfa092231fadcb76d9bd69930dce9c7f63b11fb3e7919344777c4408b7" exitCode=0 Jul 09 13:31:14.542872 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.542721 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56c6f87bc5-824bh" event={"ID":"ff1595e1-86db-4f53-a118-dff9aa8c51cc","Type":"ContainerDied","Data":"cc7c12dfa092231fadcb76d9bd69930dce9c7f63b11fb3e7919344777c4408b7"} Jul 09 13:31:14.542872 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.542733 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56c6f87bc5-824bh" Jul 09 13:31:14.542872 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.542769 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56c6f87bc5-824bh" event={"ID":"ff1595e1-86db-4f53-a118-dff9aa8c51cc","Type":"ContainerDied","Data":"592af4fe8261a2b38e13030aec00a07832f298867f104321da0b844107381dba"} Jul 09 13:31:14.542872 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.542791 2573 scope.go:117] "RemoveContainer" containerID="cc7c12dfa092231fadcb76d9bd69930dce9c7f63b11fb3e7919344777c4408b7" Jul 09 13:31:14.551932 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.551913 2573 scope.go:117] "RemoveContainer" containerID="cc7c12dfa092231fadcb76d9bd69930dce9c7f63b11fb3e7919344777c4408b7" Jul 09 13:31:14.552267 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:31:14.552241 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc7c12dfa092231fadcb76d9bd69930dce9c7f63b11fb3e7919344777c4408b7\": container with ID starting with cc7c12dfa092231fadcb76d9bd69930dce9c7f63b11fb3e7919344777c4408b7 not found: ID does not exist" containerID="cc7c12dfa092231fadcb76d9bd69930dce9c7f63b11fb3e7919344777c4408b7" Jul 09 13:31:14.552359 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.552272 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc7c12dfa092231fadcb76d9bd69930dce9c7f63b11fb3e7919344777c4408b7"} err="failed to get container status \"cc7c12dfa092231fadcb76d9bd69930dce9c7f63b11fb3e7919344777c4408b7\": rpc error: code = NotFound desc = could not find container \"cc7c12dfa092231fadcb76d9bd69930dce9c7f63b11fb3e7919344777c4408b7\": container with ID starting with cc7c12dfa092231fadcb76d9bd69930dce9c7f63b11fb3e7919344777c4408b7 not found: ID does not exist" Jul 09 13:31:14.565429 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.565392 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56c6f87bc5-824bh"] Jul 09 13:31:14.569712 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.569683 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56c6f87bc5-824bh"] Jul 09 13:31:14.945381 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:14.945349 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff1595e1-86db-4f53-a118-dff9aa8c51cc" path="/var/lib/kubelet/pods/ff1595e1-86db-4f53-a118-dff9aa8c51cc/volumes" Jul 09 13:31:19.592738 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:19.592692 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-58d464df88-gfglq" podUID="92ffc297-fe13-4928-8ea7-34e31c8c3884" containerName="console" containerID="cri-o://9b279564de5a22f06d14b869af9b3b77494eb7016ce47bb8db46d0d4459da0e1" gracePeriod=15 Jul 09 13:31:19.870071 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:19.870042 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:31:19.883010 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:19.882974 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/92ffc297-fe13-4928-8ea7-34e31c8c3884-service-ca\") pod \"92ffc297-fe13-4928-8ea7-34e31c8c3884\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " Jul 09 13:31:19.883219 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:19.883035 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/92ffc297-fe13-4928-8ea7-34e31c8c3884-oauth-serving-cert\") pod \"92ffc297-fe13-4928-8ea7-34e31c8c3884\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " Jul 09 13:31:19.883219 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:19.883066 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/92ffc297-fe13-4928-8ea7-34e31c8c3884-console-serving-cert\") pod \"92ffc297-fe13-4928-8ea7-34e31c8c3884\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " Jul 09 13:31:19.883219 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:19.883101 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/92ffc297-fe13-4928-8ea7-34e31c8c3884-console-config\") pod \"92ffc297-fe13-4928-8ea7-34e31c8c3884\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " Jul 09 13:31:19.883219 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:19.883144 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/92ffc297-fe13-4928-8ea7-34e31c8c3884-console-oauth-config\") pod \"92ffc297-fe13-4928-8ea7-34e31c8c3884\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " Jul 09 13:31:19.883219 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:19.883171 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92ffc297-fe13-4928-8ea7-34e31c8c3884-trusted-ca-bundle\") pod \"92ffc297-fe13-4928-8ea7-34e31c8c3884\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " Jul 09 13:31:19.883219 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:19.883198 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kc9sj\" (UniqueName: \"kubernetes.io/projected/92ffc297-fe13-4928-8ea7-34e31c8c3884-kube-api-access-kc9sj\") pod \"92ffc297-fe13-4928-8ea7-34e31c8c3884\" (UID: \"92ffc297-fe13-4928-8ea7-34e31c8c3884\") " Jul 09 13:31:19.883587 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:19.883560 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92ffc297-fe13-4928-8ea7-34e31c8c3884-service-ca" (OuterVolumeSpecName: "service-ca") pod "92ffc297-fe13-4928-8ea7-34e31c8c3884" (UID: "92ffc297-fe13-4928-8ea7-34e31c8c3884"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:31:19.883663 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:19.883642 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92ffc297-fe13-4928-8ea7-34e31c8c3884-console-config" (OuterVolumeSpecName: "console-config") pod "92ffc297-fe13-4928-8ea7-34e31c8c3884" (UID: "92ffc297-fe13-4928-8ea7-34e31c8c3884"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:31:19.883751 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:19.883726 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92ffc297-fe13-4928-8ea7-34e31c8c3884-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "92ffc297-fe13-4928-8ea7-34e31c8c3884" (UID: "92ffc297-fe13-4928-8ea7-34e31c8c3884"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:31:19.883812 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:19.883757 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92ffc297-fe13-4928-8ea7-34e31c8c3884-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "92ffc297-fe13-4928-8ea7-34e31c8c3884" (UID: "92ffc297-fe13-4928-8ea7-34e31c8c3884"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:31:19.886082 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:19.886050 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92ffc297-fe13-4928-8ea7-34e31c8c3884-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "92ffc297-fe13-4928-8ea7-34e31c8c3884" (UID: "92ffc297-fe13-4928-8ea7-34e31c8c3884"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:31:19.886082 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:19.886073 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92ffc297-fe13-4928-8ea7-34e31c8c3884-kube-api-access-kc9sj" (OuterVolumeSpecName: "kube-api-access-kc9sj") pod "92ffc297-fe13-4928-8ea7-34e31c8c3884" (UID: "92ffc297-fe13-4928-8ea7-34e31c8c3884"). InnerVolumeSpecName "kube-api-access-kc9sj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 13:31:19.886289 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:19.886080 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92ffc297-fe13-4928-8ea7-34e31c8c3884-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "92ffc297-fe13-4928-8ea7-34e31c8c3884" (UID: "92ffc297-fe13-4928-8ea7-34e31c8c3884"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:31:19.984613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:19.984577 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/92ffc297-fe13-4928-8ea7-34e31c8c3884-console-oauth-config\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:31:19.984613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:19.984609 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92ffc297-fe13-4928-8ea7-34e31c8c3884-trusted-ca-bundle\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:31:19.984613 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:19.984618 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kc9sj\" (UniqueName: \"kubernetes.io/projected/92ffc297-fe13-4928-8ea7-34e31c8c3884-kube-api-access-kc9sj\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:31:19.984838 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:19.984628 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/92ffc297-fe13-4928-8ea7-34e31c8c3884-service-ca\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:31:19.984838 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:19.984638 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/92ffc297-fe13-4928-8ea7-34e31c8c3884-oauth-serving-cert\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:31:19.984838 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:19.984646 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/92ffc297-fe13-4928-8ea7-34e31c8c3884-console-serving-cert\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:31:19.984838 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:19.984654 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/92ffc297-fe13-4928-8ea7-34e31c8c3884-console-config\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:31:20.578228 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:20.578195 2573 generic.go:356] "Generic (PLEG): container finished" podID="92ffc297-fe13-4928-8ea7-34e31c8c3884" containerID="9b279564de5a22f06d14b869af9b3b77494eb7016ce47bb8db46d0d4459da0e1" exitCode=0 Jul 09 13:31:20.578433 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:20.578241 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58d464df88-gfglq" Jul 09 13:31:20.578433 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:20.578282 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58d464df88-gfglq" event={"ID":"92ffc297-fe13-4928-8ea7-34e31c8c3884","Type":"ContainerDied","Data":"9b279564de5a22f06d14b869af9b3b77494eb7016ce47bb8db46d0d4459da0e1"} Jul 09 13:31:20.578433 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:20.578344 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58d464df88-gfglq" event={"ID":"92ffc297-fe13-4928-8ea7-34e31c8c3884","Type":"ContainerDied","Data":"01bc63cb8b8e4155f32e555e2274a8bc8f59b1c0abb52c72cc219b9e2f9c6714"} Jul 09 13:31:20.578433 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:20.578372 2573 scope.go:117] "RemoveContainer" containerID="9b279564de5a22f06d14b869af9b3b77494eb7016ce47bb8db46d0d4459da0e1" Jul 09 13:31:20.588470 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:20.588443 2573 scope.go:117] "RemoveContainer" containerID="9b279564de5a22f06d14b869af9b3b77494eb7016ce47bb8db46d0d4459da0e1" Jul 09 13:31:20.588842 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:31:20.588790 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b279564de5a22f06d14b869af9b3b77494eb7016ce47bb8db46d0d4459da0e1\": container with ID starting with 9b279564de5a22f06d14b869af9b3b77494eb7016ce47bb8db46d0d4459da0e1 not found: ID does not exist" containerID="9b279564de5a22f06d14b869af9b3b77494eb7016ce47bb8db46d0d4459da0e1" Jul 09 13:31:20.588913 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:20.588855 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b279564de5a22f06d14b869af9b3b77494eb7016ce47bb8db46d0d4459da0e1"} err="failed to get container status \"9b279564de5a22f06d14b869af9b3b77494eb7016ce47bb8db46d0d4459da0e1\": rpc error: code = NotFound desc = could not find container \"9b279564de5a22f06d14b869af9b3b77494eb7016ce47bb8db46d0d4459da0e1\": container with ID starting with 9b279564de5a22f06d14b869af9b3b77494eb7016ce47bb8db46d0d4459da0e1 not found: ID does not exist" Jul 09 13:31:20.603471 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:20.603440 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58d464df88-gfglq"] Jul 09 13:31:20.610426 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:20.610399 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58d464df88-gfglq"] Jul 09 13:31:20.946346 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:20.946303 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92ffc297-fe13-4928-8ea7-34e31c8c3884" path="/var/lib/kubelet/pods/92ffc297-fe13-4928-8ea7-34e31c8c3884/volumes" Jul 09 13:31:27.762252 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:27.762219 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:31:27.766553 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:27.766526 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-77764697c6-kj8fm" Jul 09 13:31:28.471361 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:28.471325 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-h6qqn" Jul 09 13:31:30.581606 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:30.581560 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6578f4b68b-pwb9x" podUID="f428915b-73cc-4e64-8d11-cdd4c4ab2038" containerName="console" containerID="cri-o://b89ab67ad58c0bf8943940b07efa711695f049c1964567aa4c2850412ce5f03c" gracePeriod=15 Jul 09 13:31:30.871437 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:30.871408 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:31:30.997218 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:30.997178 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzs54\" (UniqueName: \"kubernetes.io/projected/f428915b-73cc-4e64-8d11-cdd4c4ab2038-kube-api-access-mzs54\") pod \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\" (UID: \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\") " Jul 09 13:31:30.997380 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:30.997260 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f428915b-73cc-4e64-8d11-cdd4c4ab2038-service-ca\") pod \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\" (UID: \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\") " Jul 09 13:31:30.997380 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:30.997289 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f428915b-73cc-4e64-8d11-cdd4c4ab2038-oauth-serving-cert\") pod \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\" (UID: \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\") " Jul 09 13:31:30.997380 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:30.997323 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f428915b-73cc-4e64-8d11-cdd4c4ab2038-console-oauth-config\") pod \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\" (UID: \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\") " Jul 09 13:31:30.997380 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:30.997367 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f428915b-73cc-4e64-8d11-cdd4c4ab2038-console-serving-cert\") pod \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\" (UID: \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\") " Jul 09 13:31:30.997545 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:30.997409 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f428915b-73cc-4e64-8d11-cdd4c4ab2038-console-config\") pod \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\" (UID: \"f428915b-73cc-4e64-8d11-cdd4c4ab2038\") " Jul 09 13:31:30.997668 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:30.997640 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f428915b-73cc-4e64-8d11-cdd4c4ab2038-service-ca" (OuterVolumeSpecName: "service-ca") pod "f428915b-73cc-4e64-8d11-cdd4c4ab2038" (UID: "f428915b-73cc-4e64-8d11-cdd4c4ab2038"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:31:30.997828 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:30.997666 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f428915b-73cc-4e64-8d11-cdd4c4ab2038-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f428915b-73cc-4e64-8d11-cdd4c4ab2038" (UID: "f428915b-73cc-4e64-8d11-cdd4c4ab2038"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:31:30.997828 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:30.997784 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f428915b-73cc-4e64-8d11-cdd4c4ab2038-service-ca\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:31:30.997828 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:30.997803 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f428915b-73cc-4e64-8d11-cdd4c4ab2038-oauth-serving-cert\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:31:30.997989 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:30.997855 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f428915b-73cc-4e64-8d11-cdd4c4ab2038-console-config" (OuterVolumeSpecName: "console-config") pod "f428915b-73cc-4e64-8d11-cdd4c4ab2038" (UID: "f428915b-73cc-4e64-8d11-cdd4c4ab2038"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:31:30.999545 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:30.999519 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f428915b-73cc-4e64-8d11-cdd4c4ab2038-kube-api-access-mzs54" (OuterVolumeSpecName: "kube-api-access-mzs54") pod "f428915b-73cc-4e64-8d11-cdd4c4ab2038" (UID: "f428915b-73cc-4e64-8d11-cdd4c4ab2038"). InnerVolumeSpecName "kube-api-access-mzs54". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 13:31:30.999654 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:30.999544 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f428915b-73cc-4e64-8d11-cdd4c4ab2038-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f428915b-73cc-4e64-8d11-cdd4c4ab2038" (UID: "f428915b-73cc-4e64-8d11-cdd4c4ab2038"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:31:30.999654 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:30.999623 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f428915b-73cc-4e64-8d11-cdd4c4ab2038-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f428915b-73cc-4e64-8d11-cdd4c4ab2038" (UID: "f428915b-73cc-4e64-8d11-cdd4c4ab2038"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:31:31.098782 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:31.098685 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzs54\" (UniqueName: \"kubernetes.io/projected/f428915b-73cc-4e64-8d11-cdd4c4ab2038-kube-api-access-mzs54\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:31:31.098782 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:31.098721 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f428915b-73cc-4e64-8d11-cdd4c4ab2038-console-oauth-config\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:31:31.098782 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:31.098730 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f428915b-73cc-4e64-8d11-cdd4c4ab2038-console-serving-cert\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:31:31.098782 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:31.098739 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f428915b-73cc-4e64-8d11-cdd4c4ab2038-console-config\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:31:31.627719 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:31.627681 2573 generic.go:356] "Generic (PLEG): container finished" podID="f428915b-73cc-4e64-8d11-cdd4c4ab2038" containerID="b89ab67ad58c0bf8943940b07efa711695f049c1964567aa4c2850412ce5f03c" exitCode=0 Jul 09 13:31:31.628164 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:31.627727 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6578f4b68b-pwb9x" Jul 09 13:31:31.628164 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:31.627769 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6578f4b68b-pwb9x" event={"ID":"f428915b-73cc-4e64-8d11-cdd4c4ab2038","Type":"ContainerDied","Data":"b89ab67ad58c0bf8943940b07efa711695f049c1964567aa4c2850412ce5f03c"} Jul 09 13:31:31.628164 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:31.627823 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6578f4b68b-pwb9x" event={"ID":"f428915b-73cc-4e64-8d11-cdd4c4ab2038","Type":"ContainerDied","Data":"a7aa8f4a942cfb1a0d8d0ab43effce0f5412e2c786c81d371cc829a0cf5ccd5c"} Jul 09 13:31:31.628164 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:31.627846 2573 scope.go:117] "RemoveContainer" containerID="b89ab67ad58c0bf8943940b07efa711695f049c1964567aa4c2850412ce5f03c" Jul 09 13:31:31.636920 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:31.636903 2573 scope.go:117] "RemoveContainer" containerID="b89ab67ad58c0bf8943940b07efa711695f049c1964567aa4c2850412ce5f03c" Jul 09 13:31:31.637230 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:31:31.637204 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b89ab67ad58c0bf8943940b07efa711695f049c1964567aa4c2850412ce5f03c\": container with ID starting with b89ab67ad58c0bf8943940b07efa711695f049c1964567aa4c2850412ce5f03c not found: ID does not exist" containerID="b89ab67ad58c0bf8943940b07efa711695f049c1964567aa4c2850412ce5f03c" Jul 09 13:31:31.637315 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:31.637244 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b89ab67ad58c0bf8943940b07efa711695f049c1964567aa4c2850412ce5f03c"} err="failed to get container status \"b89ab67ad58c0bf8943940b07efa711695f049c1964567aa4c2850412ce5f03c\": rpc error: code = NotFound desc = could not find container \"b89ab67ad58c0bf8943940b07efa711695f049c1964567aa4c2850412ce5f03c\": container with ID starting with b89ab67ad58c0bf8943940b07efa711695f049c1964567aa4c2850412ce5f03c not found: ID does not exist" Jul 09 13:31:31.651533 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:31.651504 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6578f4b68b-pwb9x"] Jul 09 13:31:31.657928 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:31.657896 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6578f4b68b-pwb9x"] Jul 09 13:31:32.945278 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:32.945246 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f428915b-73cc-4e64-8d11-cdd4c4ab2038" path="/var/lib/kubelet/pods/f428915b-73cc-4e64-8d11-cdd4c4ab2038/volumes" Jul 09 13:31:49.643569 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:49.643532 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:31:49.663711 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:49.663678 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:31:49.725194 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:31:49.725166 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:03.532040 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:03.531989 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 13:32:03.766629 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:03.766580 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="alertmanager" containerID="cri-o://9b2e2c6afdc5e29cacfca2837e78db7ed1d914e5576bc6b5023a1049d2de7b61" gracePeriod=120 Jul 09 13:32:03.766840 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:03.766611 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="kube-rbac-proxy-metric" containerID="cri-o://004f34184e4b3eaaff06d5fc775fdbd7d2bbebdbaacae92821e3225297bd2e6f" gracePeriod=120 Jul 09 13:32:03.766840 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:03.766630 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="kube-rbac-proxy" containerID="cri-o://f3d316af84c32c32fb64fb24ecfab1f04fb2e2eb5766de3456573f8714bea904" gracePeriod=120 Jul 09 13:32:03.766840 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:03.766688 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="config-reloader" containerID="cri-o://b1fe1e73eacc61c6471af3e47703023d2e9b59240bd46d56c6c1de323ea37a59" gracePeriod=120 Jul 09 13:32:03.766840 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:03.766705 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="prom-label-proxy" containerID="cri-o://6af9f0d84987ed7781b4e0ba6fd8834c338a8c52c48b6ce98e30a65a203164d5" gracePeriod=120 Jul 09 13:32:03.766840 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:03.766624 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="kube-rbac-proxy-web" containerID="cri-o://e4dfda9ab8923dd0ee2fc3f4840c487a3efadfb43e83b004681fc768f29717d8" gracePeriod=120 Jul 09 13:32:04.774217 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:04.774185 2573 generic.go:356] "Generic (PLEG): container finished" podID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerID="6af9f0d84987ed7781b4e0ba6fd8834c338a8c52c48b6ce98e30a65a203164d5" exitCode=0 Jul 09 13:32:04.774217 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:04.774209 2573 generic.go:356] "Generic (PLEG): container finished" podID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerID="f3d316af84c32c32fb64fb24ecfab1f04fb2e2eb5766de3456573f8714bea904" exitCode=0 Jul 09 13:32:04.774217 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:04.774216 2573 generic.go:356] "Generic (PLEG): container finished" podID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerID="b1fe1e73eacc61c6471af3e47703023d2e9b59240bd46d56c6c1de323ea37a59" exitCode=0 Jul 09 13:32:04.774217 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:04.774222 2573 generic.go:356] "Generic (PLEG): container finished" podID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerID="9b2e2c6afdc5e29cacfca2837e78db7ed1d914e5576bc6b5023a1049d2de7b61" exitCode=0 Jul 09 13:32:04.774684 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:04.774256 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b87b16cf-44bd-4ef7-99b3-cf305030612c","Type":"ContainerDied","Data":"6af9f0d84987ed7781b4e0ba6fd8834c338a8c52c48b6ce98e30a65a203164d5"} Jul 09 13:32:04.774684 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:04.774324 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b87b16cf-44bd-4ef7-99b3-cf305030612c","Type":"ContainerDied","Data":"f3d316af84c32c32fb64fb24ecfab1f04fb2e2eb5766de3456573f8714bea904"} Jul 09 13:32:04.774684 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:04.774350 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b87b16cf-44bd-4ef7-99b3-cf305030612c","Type":"ContainerDied","Data":"b1fe1e73eacc61c6471af3e47703023d2e9b59240bd46d56c6c1de323ea37a59"} Jul 09 13:32:04.774684 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:04.774375 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b87b16cf-44bd-4ef7-99b3-cf305030612c","Type":"ContainerDied","Data":"9b2e2c6afdc5e29cacfca2837e78db7ed1d914e5576bc6b5023a1049d2de7b61"} Jul 09 13:32:05.017499 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.017470 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:05.101947 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.101856 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-config-volume\") pod \"b87b16cf-44bd-4ef7-99b3-cf305030612c\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " Jul 09 13:32:05.101947 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.101898 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-secret-alertmanager-kube-rbac-proxy-web\") pod \"b87b16cf-44bd-4ef7-99b3-cf305030612c\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " Jul 09 13:32:05.101947 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.101928 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgc82\" (UniqueName: \"kubernetes.io/projected/b87b16cf-44bd-4ef7-99b3-cf305030612c-kube-api-access-sgc82\") pod \"b87b16cf-44bd-4ef7-99b3-cf305030612c\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " Jul 09 13:32:05.102262 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.101951 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b87b16cf-44bd-4ef7-99b3-cf305030612c-metrics-client-ca\") pod \"b87b16cf-44bd-4ef7-99b3-cf305030612c\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " Jul 09 13:32:05.102262 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.101983 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b87b16cf-44bd-4ef7-99b3-cf305030612c-config-out\") pod \"b87b16cf-44bd-4ef7-99b3-cf305030612c\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " Jul 09 13:32:05.102378 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.102345 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-secret-alertmanager-main-tls\") pod \"b87b16cf-44bd-4ef7-99b3-cf305030612c\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " Jul 09 13:32:05.102492 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.102474 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b87b16cf-44bd-4ef7-99b3-cf305030612c-tls-assets\") pod \"b87b16cf-44bd-4ef7-99b3-cf305030612c\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " Jul 09 13:32:05.102573 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.102524 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-cluster-tls-config\") pod \"b87b16cf-44bd-4ef7-99b3-cf305030612c\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " Jul 09 13:32:05.102573 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.102555 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-web-config\") pod \"b87b16cf-44bd-4ef7-99b3-cf305030612c\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " Jul 09 13:32:05.102573 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.102548 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b87b16cf-44bd-4ef7-99b3-cf305030612c-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "b87b16cf-44bd-4ef7-99b3-cf305030612c" (UID: "b87b16cf-44bd-4ef7-99b3-cf305030612c"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:32:05.102728 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.102580 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"b87b16cf-44bd-4ef7-99b3-cf305030612c\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " Jul 09 13:32:05.102728 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.102637 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-secret-alertmanager-kube-rbac-proxy\") pod \"b87b16cf-44bd-4ef7-99b3-cf305030612c\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " Jul 09 13:32:05.102728 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.102680 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b87b16cf-44bd-4ef7-99b3-cf305030612c-alertmanager-trusted-ca-bundle\") pod \"b87b16cf-44bd-4ef7-99b3-cf305030612c\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " Jul 09 13:32:05.102873 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.102737 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b87b16cf-44bd-4ef7-99b3-cf305030612c-alertmanager-main-db\") pod \"b87b16cf-44bd-4ef7-99b3-cf305030612c\" (UID: \"b87b16cf-44bd-4ef7-99b3-cf305030612c\") " Jul 09 13:32:05.103459 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.103058 2573 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b87b16cf-44bd-4ef7-99b3-cf305030612c-metrics-client-ca\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:05.103459 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.103343 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b87b16cf-44bd-4ef7-99b3-cf305030612c-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "b87b16cf-44bd-4ef7-99b3-cf305030612c" (UID: "b87b16cf-44bd-4ef7-99b3-cf305030612c"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:32:05.103459 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.103409 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b87b16cf-44bd-4ef7-99b3-cf305030612c-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "b87b16cf-44bd-4ef7-99b3-cf305030612c" (UID: "b87b16cf-44bd-4ef7-99b3-cf305030612c"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:32:05.106769 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.106631 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-config-volume" (OuterVolumeSpecName: "config-volume") pod "b87b16cf-44bd-4ef7-99b3-cf305030612c" (UID: "b87b16cf-44bd-4ef7-99b3-cf305030612c"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:32:05.107299 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.107243 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "b87b16cf-44bd-4ef7-99b3-cf305030612c" (UID: "b87b16cf-44bd-4ef7-99b3-cf305030612c"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:32:05.107445 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.107376 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b87b16cf-44bd-4ef7-99b3-cf305030612c-kube-api-access-sgc82" (OuterVolumeSpecName: "kube-api-access-sgc82") pod "b87b16cf-44bd-4ef7-99b3-cf305030612c" (UID: "b87b16cf-44bd-4ef7-99b3-cf305030612c"). InnerVolumeSpecName "kube-api-access-sgc82". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 13:32:05.107523 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.107494 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "b87b16cf-44bd-4ef7-99b3-cf305030612c" (UID: "b87b16cf-44bd-4ef7-99b3-cf305030612c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:32:05.107752 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.107724 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "b87b16cf-44bd-4ef7-99b3-cf305030612c" (UID: "b87b16cf-44bd-4ef7-99b3-cf305030612c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:32:05.107863 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.107768 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b87b16cf-44bd-4ef7-99b3-cf305030612c-config-out" (OuterVolumeSpecName: "config-out") pod "b87b16cf-44bd-4ef7-99b3-cf305030612c" (UID: "b87b16cf-44bd-4ef7-99b3-cf305030612c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:32:05.107932 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.107878 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "b87b16cf-44bd-4ef7-99b3-cf305030612c" (UID: "b87b16cf-44bd-4ef7-99b3-cf305030612c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:32:05.107932 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.107901 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b87b16cf-44bd-4ef7-99b3-cf305030612c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "b87b16cf-44bd-4ef7-99b3-cf305030612c" (UID: "b87b16cf-44bd-4ef7-99b3-cf305030612c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 13:32:05.112126 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.112014 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "b87b16cf-44bd-4ef7-99b3-cf305030612c" (UID: "b87b16cf-44bd-4ef7-99b3-cf305030612c"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:32:05.120123 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.120001 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-web-config" (OuterVolumeSpecName: "web-config") pod "b87b16cf-44bd-4ef7-99b3-cf305030612c" (UID: "b87b16cf-44bd-4ef7-99b3-cf305030612c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:32:05.203722 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.203684 2573 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-cluster-tls-config\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:05.203722 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.203713 2573 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-web-config\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:05.203722 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.203724 2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:05.203950 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.203734 2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:05.203950 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.203745 2573 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b87b16cf-44bd-4ef7-99b3-cf305030612c-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:05.203950 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.203754 2573 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b87b16cf-44bd-4ef7-99b3-cf305030612c-alertmanager-main-db\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:05.203950 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.203762 2573 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-config-volume\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:05.203950 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.203770 2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:05.203950 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.203782 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgc82\" (UniqueName: \"kubernetes.io/projected/b87b16cf-44bd-4ef7-99b3-cf305030612c-kube-api-access-sgc82\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:05.203950 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.203789 2573 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b87b16cf-44bd-4ef7-99b3-cf305030612c-config-out\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:05.203950 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.203798 2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b87b16cf-44bd-4ef7-99b3-cf305030612c-secret-alertmanager-main-tls\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:05.203950 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.203805 2573 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b87b16cf-44bd-4ef7-99b3-cf305030612c-tls-assets\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:05.542232 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542197 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5665b7b4df-6skjd"] Jul 09 13:32:05.542614 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542588 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f428915b-73cc-4e64-8d11-cdd4c4ab2038" containerName="console" Jul 09 13:32:05.542614 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542607 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f428915b-73cc-4e64-8d11-cdd4c4ab2038" containerName="console" Jul 09 13:32:05.542765 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542626 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="kube-rbac-proxy" Jul 09 13:32:05.542765 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542635 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="kube-rbac-proxy" Jul 09 13:32:05.542765 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542657 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff1595e1-86db-4f53-a118-dff9aa8c51cc" containerName="console" Jul 09 13:32:05.542765 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542665 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff1595e1-86db-4f53-a118-dff9aa8c51cc" containerName="console" Jul 09 13:32:05.542765 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542686 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="alertmanager" Jul 09 13:32:05.542765 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542694 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="alertmanager" Jul 09 13:32:05.542765 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542705 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92ffc297-fe13-4928-8ea7-34e31c8c3884" containerName="console" Jul 09 13:32:05.542765 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542712 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="92ffc297-fe13-4928-8ea7-34e31c8c3884" containerName="console" Jul 09 13:32:05.542765 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542728 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="config-reloader" Jul 09 13:32:05.542765 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542737 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="config-reloader" Jul 09 13:32:05.542765 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542746 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="kube-rbac-proxy-web" Jul 09 13:32:05.542765 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542755 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="kube-rbac-proxy-web" Jul 09 13:32:05.542765 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542764 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="kube-rbac-proxy-metric" Jul 09 13:32:05.543380 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542772 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="kube-rbac-proxy-metric" Jul 09 13:32:05.543380 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542783 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="init-config-reloader" Jul 09 13:32:05.543380 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542792 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="init-config-reloader" Jul 09 13:32:05.543380 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542800 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="prom-label-proxy" Jul 09 13:32:05.543380 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542809 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="prom-label-proxy" Jul 09 13:32:05.543380 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542887 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="kube-rbac-proxy-metric" Jul 09 13:32:05.543380 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542902 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="prom-label-proxy" Jul 09 13:32:05.543380 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542912 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="kube-rbac-proxy" Jul 09 13:32:05.543380 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542922 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="alertmanager" Jul 09 13:32:05.543380 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542931 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="config-reloader" Jul 09 13:32:05.543380 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542940 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerName="kube-rbac-proxy-web" Jul 09 13:32:05.543380 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542949 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f428915b-73cc-4e64-8d11-cdd4c4ab2038" containerName="console" Jul 09 13:32:05.543380 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542958 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff1595e1-86db-4f53-a118-dff9aa8c51cc" containerName="console" Jul 09 13:32:05.543380 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.542965 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="92ffc297-fe13-4928-8ea7-34e31c8c3884" containerName="console" Jul 09 13:32:05.546147 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.546124 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:05.578402 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.578364 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5665b7b4df-6skjd"] Jul 09 13:32:05.607378 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.607339 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-console-config\") pod \"console-5665b7b4df-6skjd\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:05.607378 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.607377 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-service-ca\") pod \"console-5665b7b4df-6skjd\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:05.607589 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.607409 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-console-oauth-config\") pod \"console-5665b7b4df-6skjd\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:05.607589 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.607434 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4shzd\" (UniqueName: \"kubernetes.io/projected/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-kube-api-access-4shzd\") pod \"console-5665b7b4df-6skjd\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:05.607589 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.607472 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-trusted-ca-bundle\") pod \"console-5665b7b4df-6skjd\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:05.607589 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.607514 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-console-serving-cert\") pod \"console-5665b7b4df-6skjd\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:05.607589 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.607534 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-oauth-serving-cert\") pod \"console-5665b7b4df-6skjd\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:05.708473 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.708438 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-trusted-ca-bundle\") pod \"console-5665b7b4df-6skjd\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:05.708671 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.708487 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-console-serving-cert\") pod \"console-5665b7b4df-6skjd\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:05.708671 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.708514 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-oauth-serving-cert\") pod \"console-5665b7b4df-6skjd\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:05.708671 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.708556 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-console-config\") pod \"console-5665b7b4df-6skjd\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:05.708671 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.708586 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-service-ca\") pod \"console-5665b7b4df-6skjd\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:05.708671 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.708647 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-console-oauth-config\") pod \"console-5665b7b4df-6skjd\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:05.708915 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.708682 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4shzd\" (UniqueName: \"kubernetes.io/projected/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-kube-api-access-4shzd\") pod \"console-5665b7b4df-6skjd\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:05.709390 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.709360 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-console-config\") pod \"console-5665b7b4df-6skjd\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:05.709509 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.709443 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-oauth-serving-cert\") pod \"console-5665b7b4df-6skjd\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:05.709547 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.709514 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-service-ca\") pod \"console-5665b7b4df-6skjd\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:05.709547 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.709537 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-trusted-ca-bundle\") pod \"console-5665b7b4df-6skjd\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:05.710974 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.710948 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-console-serving-cert\") pod \"console-5665b7b4df-6skjd\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:05.711085 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.710981 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-console-oauth-config\") pod \"console-5665b7b4df-6skjd\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:05.729303 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.729270 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4shzd\" (UniqueName: \"kubernetes.io/projected/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-kube-api-access-4shzd\") pod \"console-5665b7b4df-6skjd\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:05.787598 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.787557 2573 generic.go:356] "Generic (PLEG): container finished" podID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerID="004f34184e4b3eaaff06d5fc775fdbd7d2bbebdbaacae92821e3225297bd2e6f" exitCode=0 Jul 09 13:32:05.787598 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.787589 2573 generic.go:356] "Generic (PLEG): container finished" podID="b87b16cf-44bd-4ef7-99b3-cf305030612c" containerID="e4dfda9ab8923dd0ee2fc3f4840c487a3efadfb43e83b004681fc768f29717d8" exitCode=0 Jul 09 13:32:05.788136 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.787624 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b87b16cf-44bd-4ef7-99b3-cf305030612c","Type":"ContainerDied","Data":"004f34184e4b3eaaff06d5fc775fdbd7d2bbebdbaacae92821e3225297bd2e6f"} Jul 09 13:32:05.788136 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.787647 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:05.788136 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.787742 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b87b16cf-44bd-4ef7-99b3-cf305030612c","Type":"ContainerDied","Data":"e4dfda9ab8923dd0ee2fc3f4840c487a3efadfb43e83b004681fc768f29717d8"} Jul 09 13:32:05.788136 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.787787 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b87b16cf-44bd-4ef7-99b3-cf305030612c","Type":"ContainerDied","Data":"8382057708fd427098f416665f8766b5965e319595ddc60fbf1fe33f597bf9b2"} Jul 09 13:32:05.788136 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.787813 2573 scope.go:117] "RemoveContainer" containerID="6af9f0d84987ed7781b4e0ba6fd8834c338a8c52c48b6ce98e30a65a203164d5" Jul 09 13:32:05.796760 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.796738 2573 scope.go:117] "RemoveContainer" containerID="004f34184e4b3eaaff06d5fc775fdbd7d2bbebdbaacae92821e3225297bd2e6f" Jul 09 13:32:05.804404 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.804360 2573 scope.go:117] "RemoveContainer" containerID="f3d316af84c32c32fb64fb24ecfab1f04fb2e2eb5766de3456573f8714bea904" Jul 09 13:32:05.812088 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.812071 2573 scope.go:117] "RemoveContainer" containerID="e4dfda9ab8923dd0ee2fc3f4840c487a3efadfb43e83b004681fc768f29717d8" Jul 09 13:32:05.819864 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.819844 2573 scope.go:117] "RemoveContainer" containerID="b1fe1e73eacc61c6471af3e47703023d2e9b59240bd46d56c6c1de323ea37a59" Jul 09 13:32:05.827915 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.827896 2573 scope.go:117] "RemoveContainer" containerID="9b2e2c6afdc5e29cacfca2837e78db7ed1d914e5576bc6b5023a1049d2de7b61" Jul 09 13:32:05.835867 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.835845 2573 scope.go:117] "RemoveContainer" containerID="72978e0a2c65aa9a24844d253eef0c8ff4645cbd9e4491a876882252b6617d5e" Jul 09 13:32:05.843142 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.843116 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 13:32:05.852760 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.852734 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 13:32:05.854957 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.854938 2573 scope.go:117] "RemoveContainer" containerID="6af9f0d84987ed7781b4e0ba6fd8834c338a8c52c48b6ce98e30a65a203164d5" Jul 09 13:32:05.855109 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.855093 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:05.855329 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:32:05.855310 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6af9f0d84987ed7781b4e0ba6fd8834c338a8c52c48b6ce98e30a65a203164d5\": container with ID starting with 6af9f0d84987ed7781b4e0ba6fd8834c338a8c52c48b6ce98e30a65a203164d5 not found: ID does not exist" containerID="6af9f0d84987ed7781b4e0ba6fd8834c338a8c52c48b6ce98e30a65a203164d5" Jul 09 13:32:05.855387 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.855338 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6af9f0d84987ed7781b4e0ba6fd8834c338a8c52c48b6ce98e30a65a203164d5"} err="failed to get container status \"6af9f0d84987ed7781b4e0ba6fd8834c338a8c52c48b6ce98e30a65a203164d5\": rpc error: code = NotFound desc = could not find container \"6af9f0d84987ed7781b4e0ba6fd8834c338a8c52c48b6ce98e30a65a203164d5\": container with ID starting with 6af9f0d84987ed7781b4e0ba6fd8834c338a8c52c48b6ce98e30a65a203164d5 not found: ID does not exist" Jul 09 13:32:05.855387 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.855356 2573 scope.go:117] "RemoveContainer" containerID="004f34184e4b3eaaff06d5fc775fdbd7d2bbebdbaacae92821e3225297bd2e6f" Jul 09 13:32:05.855654 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:32:05.855636 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"004f34184e4b3eaaff06d5fc775fdbd7d2bbebdbaacae92821e3225297bd2e6f\": container with ID starting with 004f34184e4b3eaaff06d5fc775fdbd7d2bbebdbaacae92821e3225297bd2e6f not found: ID does not exist" containerID="004f34184e4b3eaaff06d5fc775fdbd7d2bbebdbaacae92821e3225297bd2e6f" Jul 09 13:32:05.855722 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.855665 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"004f34184e4b3eaaff06d5fc775fdbd7d2bbebdbaacae92821e3225297bd2e6f"} err="failed to get container status \"004f34184e4b3eaaff06d5fc775fdbd7d2bbebdbaacae92821e3225297bd2e6f\": rpc error: code = NotFound desc = could not find container \"004f34184e4b3eaaff06d5fc775fdbd7d2bbebdbaacae92821e3225297bd2e6f\": container with ID starting with 004f34184e4b3eaaff06d5fc775fdbd7d2bbebdbaacae92821e3225297bd2e6f not found: ID does not exist" Jul 09 13:32:05.855722 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.855687 2573 scope.go:117] "RemoveContainer" containerID="f3d316af84c32c32fb64fb24ecfab1f04fb2e2eb5766de3456573f8714bea904" Jul 09 13:32:05.855942 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:32:05.855926 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3d316af84c32c32fb64fb24ecfab1f04fb2e2eb5766de3456573f8714bea904\": container with ID starting with f3d316af84c32c32fb64fb24ecfab1f04fb2e2eb5766de3456573f8714bea904 not found: ID does not exist" containerID="f3d316af84c32c32fb64fb24ecfab1f04fb2e2eb5766de3456573f8714bea904" Jul 09 13:32:05.855978 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.855955 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3d316af84c32c32fb64fb24ecfab1f04fb2e2eb5766de3456573f8714bea904"} err="failed to get container status \"f3d316af84c32c32fb64fb24ecfab1f04fb2e2eb5766de3456573f8714bea904\": rpc error: code = NotFound desc = could not find container \"f3d316af84c32c32fb64fb24ecfab1f04fb2e2eb5766de3456573f8714bea904\": container with ID starting with f3d316af84c32c32fb64fb24ecfab1f04fb2e2eb5766de3456573f8714bea904 not found: ID does not exist" Jul 09 13:32:05.855978 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.855970 2573 scope.go:117] "RemoveContainer" containerID="e4dfda9ab8923dd0ee2fc3f4840c487a3efadfb43e83b004681fc768f29717d8" Jul 09 13:32:05.856229 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:32:05.856210 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4dfda9ab8923dd0ee2fc3f4840c487a3efadfb43e83b004681fc768f29717d8\": container with ID starting with e4dfda9ab8923dd0ee2fc3f4840c487a3efadfb43e83b004681fc768f29717d8 not found: ID does not exist" containerID="e4dfda9ab8923dd0ee2fc3f4840c487a3efadfb43e83b004681fc768f29717d8" Jul 09 13:32:05.856294 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.856242 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4dfda9ab8923dd0ee2fc3f4840c487a3efadfb43e83b004681fc768f29717d8"} err="failed to get container status \"e4dfda9ab8923dd0ee2fc3f4840c487a3efadfb43e83b004681fc768f29717d8\": rpc error: code = NotFound desc = could not find container \"e4dfda9ab8923dd0ee2fc3f4840c487a3efadfb43e83b004681fc768f29717d8\": container with ID starting with e4dfda9ab8923dd0ee2fc3f4840c487a3efadfb43e83b004681fc768f29717d8 not found: ID does not exist" Jul 09 13:32:05.856294 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.856262 2573 scope.go:117] "RemoveContainer" containerID="b1fe1e73eacc61c6471af3e47703023d2e9b59240bd46d56c6c1de323ea37a59" Jul 09 13:32:05.856527 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:32:05.856508 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1fe1e73eacc61c6471af3e47703023d2e9b59240bd46d56c6c1de323ea37a59\": container with ID starting with b1fe1e73eacc61c6471af3e47703023d2e9b59240bd46d56c6c1de323ea37a59 not found: ID does not exist" containerID="b1fe1e73eacc61c6471af3e47703023d2e9b59240bd46d56c6c1de323ea37a59" Jul 09 13:32:05.856578 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.856531 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1fe1e73eacc61c6471af3e47703023d2e9b59240bd46d56c6c1de323ea37a59"} err="failed to get container status \"b1fe1e73eacc61c6471af3e47703023d2e9b59240bd46d56c6c1de323ea37a59\": rpc error: code = NotFound desc = could not find container \"b1fe1e73eacc61c6471af3e47703023d2e9b59240bd46d56c6c1de323ea37a59\": container with ID starting with b1fe1e73eacc61c6471af3e47703023d2e9b59240bd46d56c6c1de323ea37a59 not found: ID does not exist" Jul 09 13:32:05.856578 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.856548 2573 scope.go:117] "RemoveContainer" containerID="9b2e2c6afdc5e29cacfca2837e78db7ed1d914e5576bc6b5023a1049d2de7b61" Jul 09 13:32:05.856792 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:32:05.856776 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b2e2c6afdc5e29cacfca2837e78db7ed1d914e5576bc6b5023a1049d2de7b61\": container with ID starting with 9b2e2c6afdc5e29cacfca2837e78db7ed1d914e5576bc6b5023a1049d2de7b61 not found: ID does not exist" containerID="9b2e2c6afdc5e29cacfca2837e78db7ed1d914e5576bc6b5023a1049d2de7b61" Jul 09 13:32:05.856831 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.856796 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b2e2c6afdc5e29cacfca2837e78db7ed1d914e5576bc6b5023a1049d2de7b61"} err="failed to get container status \"9b2e2c6afdc5e29cacfca2837e78db7ed1d914e5576bc6b5023a1049d2de7b61\": rpc error: code = NotFound desc = could not find container \"9b2e2c6afdc5e29cacfca2837e78db7ed1d914e5576bc6b5023a1049d2de7b61\": container with ID starting with 9b2e2c6afdc5e29cacfca2837e78db7ed1d914e5576bc6b5023a1049d2de7b61 not found: ID does not exist" Jul 09 13:32:05.856831 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.856819 2573 scope.go:117] "RemoveContainer" containerID="72978e0a2c65aa9a24844d253eef0c8ff4645cbd9e4491a876882252b6617d5e" Jul 09 13:32:05.857059 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:32:05.857042 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72978e0a2c65aa9a24844d253eef0c8ff4645cbd9e4491a876882252b6617d5e\": container with ID starting with 72978e0a2c65aa9a24844d253eef0c8ff4645cbd9e4491a876882252b6617d5e not found: ID does not exist" containerID="72978e0a2c65aa9a24844d253eef0c8ff4645cbd9e4491a876882252b6617d5e" Jul 09 13:32:05.857108 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.857063 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72978e0a2c65aa9a24844d253eef0c8ff4645cbd9e4491a876882252b6617d5e"} err="failed to get container status \"72978e0a2c65aa9a24844d253eef0c8ff4645cbd9e4491a876882252b6617d5e\": rpc error: code = NotFound desc = could not find container \"72978e0a2c65aa9a24844d253eef0c8ff4645cbd9e4491a876882252b6617d5e\": container with ID starting with 72978e0a2c65aa9a24844d253eef0c8ff4645cbd9e4491a876882252b6617d5e not found: ID does not exist" Jul 09 13:32:05.857108 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.857077 2573 scope.go:117] "RemoveContainer" containerID="6af9f0d84987ed7781b4e0ba6fd8834c338a8c52c48b6ce98e30a65a203164d5" Jul 09 13:32:05.857332 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.857312 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6af9f0d84987ed7781b4e0ba6fd8834c338a8c52c48b6ce98e30a65a203164d5"} err="failed to get container status \"6af9f0d84987ed7781b4e0ba6fd8834c338a8c52c48b6ce98e30a65a203164d5\": rpc error: code = NotFound desc = could not find container \"6af9f0d84987ed7781b4e0ba6fd8834c338a8c52c48b6ce98e30a65a203164d5\": container with ID starting with 6af9f0d84987ed7781b4e0ba6fd8834c338a8c52c48b6ce98e30a65a203164d5 not found: ID does not exist" Jul 09 13:32:05.857332 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.857332 2573 scope.go:117] "RemoveContainer" containerID="004f34184e4b3eaaff06d5fc775fdbd7d2bbebdbaacae92821e3225297bd2e6f" Jul 09 13:32:05.857559 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.857543 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"004f34184e4b3eaaff06d5fc775fdbd7d2bbebdbaacae92821e3225297bd2e6f"} err="failed to get container status \"004f34184e4b3eaaff06d5fc775fdbd7d2bbebdbaacae92821e3225297bd2e6f\": rpc error: code = NotFound desc = could not find container \"004f34184e4b3eaaff06d5fc775fdbd7d2bbebdbaacae92821e3225297bd2e6f\": container with ID starting with 004f34184e4b3eaaff06d5fc775fdbd7d2bbebdbaacae92821e3225297bd2e6f not found: ID does not exist" Jul 09 13:32:05.857611 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.857560 2573 scope.go:117] "RemoveContainer" containerID="f3d316af84c32c32fb64fb24ecfab1f04fb2e2eb5766de3456573f8714bea904" Jul 09 13:32:05.857769 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.857752 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3d316af84c32c32fb64fb24ecfab1f04fb2e2eb5766de3456573f8714bea904"} err="failed to get container status \"f3d316af84c32c32fb64fb24ecfab1f04fb2e2eb5766de3456573f8714bea904\": rpc error: code = NotFound desc = could not find container \"f3d316af84c32c32fb64fb24ecfab1f04fb2e2eb5766de3456573f8714bea904\": container with ID starting with f3d316af84c32c32fb64fb24ecfab1f04fb2e2eb5766de3456573f8714bea904 not found: ID does not exist" Jul 09 13:32:05.857809 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.857769 2573 scope.go:117] "RemoveContainer" containerID="e4dfda9ab8923dd0ee2fc3f4840c487a3efadfb43e83b004681fc768f29717d8" Jul 09 13:32:05.858070 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.858051 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4dfda9ab8923dd0ee2fc3f4840c487a3efadfb43e83b004681fc768f29717d8"} err="failed to get container status \"e4dfda9ab8923dd0ee2fc3f4840c487a3efadfb43e83b004681fc768f29717d8\": rpc error: code = NotFound desc = could not find container \"e4dfda9ab8923dd0ee2fc3f4840c487a3efadfb43e83b004681fc768f29717d8\": container with ID starting with e4dfda9ab8923dd0ee2fc3f4840c487a3efadfb43e83b004681fc768f29717d8 not found: ID does not exist" Jul 09 13:32:05.858131 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.858071 2573 scope.go:117] "RemoveContainer" containerID="b1fe1e73eacc61c6471af3e47703023d2e9b59240bd46d56c6c1de323ea37a59" Jul 09 13:32:05.858301 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.858279 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1fe1e73eacc61c6471af3e47703023d2e9b59240bd46d56c6c1de323ea37a59"} err="failed to get container status \"b1fe1e73eacc61c6471af3e47703023d2e9b59240bd46d56c6c1de323ea37a59\": rpc error: code = NotFound desc = could not find container \"b1fe1e73eacc61c6471af3e47703023d2e9b59240bd46d56c6c1de323ea37a59\": container with ID starting with b1fe1e73eacc61c6471af3e47703023d2e9b59240bd46d56c6c1de323ea37a59 not found: ID does not exist" Jul 09 13:32:05.858347 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.858303 2573 scope.go:117] "RemoveContainer" containerID="9b2e2c6afdc5e29cacfca2837e78db7ed1d914e5576bc6b5023a1049d2de7b61" Jul 09 13:32:05.858546 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.858527 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b2e2c6afdc5e29cacfca2837e78db7ed1d914e5576bc6b5023a1049d2de7b61"} err="failed to get container status \"9b2e2c6afdc5e29cacfca2837e78db7ed1d914e5576bc6b5023a1049d2de7b61\": rpc error: code = NotFound desc = could not find container \"9b2e2c6afdc5e29cacfca2837e78db7ed1d914e5576bc6b5023a1049d2de7b61\": container with ID starting with 9b2e2c6afdc5e29cacfca2837e78db7ed1d914e5576bc6b5023a1049d2de7b61 not found: ID does not exist" Jul 09 13:32:05.858619 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.858547 2573 scope.go:117] "RemoveContainer" containerID="72978e0a2c65aa9a24844d253eef0c8ff4645cbd9e4491a876882252b6617d5e" Jul 09 13:32:05.858890 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.858805 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72978e0a2c65aa9a24844d253eef0c8ff4645cbd9e4491a876882252b6617d5e"} err="failed to get container status \"72978e0a2c65aa9a24844d253eef0c8ff4645cbd9e4491a876882252b6617d5e\": rpc error: code = NotFound desc = could not find container \"72978e0a2c65aa9a24844d253eef0c8ff4645cbd9e4491a876882252b6617d5e\": container with ID starting with 72978e0a2c65aa9a24844d253eef0c8ff4645cbd9e4491a876882252b6617d5e not found: ID does not exist" Jul 09 13:32:05.887524 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.887475 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 13:32:05.891105 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.891078 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:05.895570 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.895509 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 09 13:32:05.895570 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.895537 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 09 13:32:05.895803 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.895606 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-mbtzb\"" Jul 09 13:32:05.895803 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.895537 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 09 13:32:05.895803 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.895509 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 09 13:32:05.896061 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.895924 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 09 13:32:05.896061 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.895993 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 09 13:32:05.896277 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.896241 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 09 13:32:05.896277 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.896266 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 09 13:32:05.898952 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.898928 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 09 13:32:05.912551 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:05.912341 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 13:32:06.007384 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.007357 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5665b7b4df-6skjd"] Jul 09 13:32:06.009979 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:32:06.009952 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ed7d06a_8a12_401f_98c1_3e26f38d6c50.slice/crio-8ba66ee4f5135aec0bed3ff2794f965c41292fb5d5d49634f91fe4d73b8217de WatchSource:0}: Error finding container 8ba66ee4f5135aec0bed3ff2794f965c41292fb5d5d49634f91fe4d73b8217de: Status 404 returned error can't find the container with id 8ba66ee4f5135aec0bed3ff2794f965c41292fb5d5d49634f91fe4d73b8217de Jul 09 13:32:06.010921 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.010898 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.011013 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.010933 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.011013 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.010961 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9cfr\" (UniqueName: \"kubernetes.io/projected/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-kube-api-access-l9cfr\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.011013 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.011014 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.011186 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.011079 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.011186 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.011162 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.011276 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.011213 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-web-config\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.011373 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.011353 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.011432 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.011389 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.011486 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.011423 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-config-volume\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.011486 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.011454 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.011578 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.011490 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-config-out\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.011578 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.011518 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.112954 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.112876 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-config-volume\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.112954 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.112934 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.113226 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.112965 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-config-out\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.113226 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.112995 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.113759 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.113727 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.113878 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.113777 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.113878 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.113806 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9cfr\" (UniqueName: \"kubernetes.io/projected/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-kube-api-access-l9cfr\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.113878 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.113839 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.113878 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.113873 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.114138 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.113909 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.114138 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.113947 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-web-config\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.114138 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.113988 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.114138 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.114056 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.114349 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.114140 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.114839 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.114799 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.115207 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.115180 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.116884 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.116838 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.117654 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.117132 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.117654 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.117202 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-config-volume\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.117654 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.117208 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-config-out\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.117654 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.117613 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.117930 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.117674 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.118305 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.118277 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.118447 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.118428 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.118647 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.118632 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-web-config\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.129090 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.129006 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9cfr\" (UniqueName: \"kubernetes.io/projected/d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f-kube-api-access-l9cfr\") pod \"alertmanager-main-0\" (UID: \"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.205626 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.205593 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 13:32:06.358948 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.358922 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 13:32:06.360917 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:32:06.360888 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7f311ed_6ab6_4fd3_87b8_bfc6c71a0a6f.slice/crio-fecbc0fc74da1716594848213f74a33ac53672e461c7b59f43c456c6796a266e WatchSource:0}: Error finding container fecbc0fc74da1716594848213f74a33ac53672e461c7b59f43c456c6796a266e: Status 404 returned error can't find the container with id fecbc0fc74da1716594848213f74a33ac53672e461c7b59f43c456c6796a266e Jul 09 13:32:06.794823 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.794789 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5665b7b4df-6skjd" event={"ID":"2ed7d06a-8a12-401f-98c1-3e26f38d6c50","Type":"ContainerStarted","Data":"a4756037ea0a5c3017bcdd47355d8af7780cdbf07e72f4b6e457f8df2a992597"} Jul 09 13:32:06.795287 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.794835 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5665b7b4df-6skjd" event={"ID":"2ed7d06a-8a12-401f-98c1-3e26f38d6c50","Type":"ContainerStarted","Data":"8ba66ee4f5135aec0bed3ff2794f965c41292fb5d5d49634f91fe4d73b8217de"} Jul 09 13:32:06.796378 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.796353 2573 generic.go:356] "Generic (PLEG): container finished" podID="d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f" containerID="55f3e909d5d268179e9afd9cea7b2a1cc0e8ffd31cc7c0fcd75a00e4c20a5cc0" exitCode=0 Jul 09 13:32:06.796488 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.796442 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f","Type":"ContainerDied","Data":"55f3e909d5d268179e9afd9cea7b2a1cc0e8ffd31cc7c0fcd75a00e4c20a5cc0"} Jul 09 13:32:06.796588 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.796512 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f","Type":"ContainerStarted","Data":"fecbc0fc74da1716594848213f74a33ac53672e461c7b59f43c456c6796a266e"} Jul 09 13:32:06.816718 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.816667 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5665b7b4df-6skjd" podStartSLOduration=1.816651587 podStartE2EDuration="1.816651587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:32:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:32:06.815326253 +0000 UTC m=+144.320628520" watchObservedRunningTime="2026-07-09 13:32:06.816651587 +0000 UTC m=+144.321953855" Jul 09 13:32:06.946345 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:06.946312 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b87b16cf-44bd-4ef7-99b3-cf305030612c" path="/var/lib/kubelet/pods/b87b16cf-44bd-4ef7-99b3-cf305030612c/volumes" Jul 09 13:32:07.811966 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:07.811926 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f","Type":"ContainerStarted","Data":"98d9e7aad3a4ee31f960f85e1ff766d75d4939929bfa0ade8fcc27d9b034e043"} Jul 09 13:32:07.812462 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:07.812013 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f","Type":"ContainerStarted","Data":"3854d961fdfcf5d297ed96c95076b72d5ad7ef7ba6e914600a2fb902873b9ce7"} Jul 09 13:32:07.812462 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:07.812087 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f","Type":"ContainerStarted","Data":"decc9ec33722f68f07ba9c6cb38984f72f664c5c1c19c6268362d69d45297eea"} Jul 09 13:32:07.812462 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:07.812127 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f","Type":"ContainerStarted","Data":"9c82044edc65e2113bc4409cd9cf5f07bc45be44869c78b7cf0709850b460cc8"} Jul 09 13:32:07.812462 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:07.812162 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f","Type":"ContainerStarted","Data":"4c8bdf86730c4ca6b4d526e4ca7edbce903d97d9d968fa156277566007130c3f"} Jul 09 13:32:07.812462 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:07.812199 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f","Type":"ContainerStarted","Data":"34cbc514b5654a13e2a3ebcd7dfac5147a1662f5714978ef5231954501267a93"} Jul 09 13:32:07.812980 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:07.812956 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 13:32:07.813291 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:07.813263 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="prometheus" containerID="cri-o://b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c" gracePeriod=600 Jul 09 13:32:07.813291 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:07.813283 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="kube-rbac-proxy" containerID="cri-o://d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae" gracePeriod=600 Jul 09 13:32:07.813431 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:07.813291 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="thanos-sidecar" containerID="cri-o://abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784" gracePeriod=600 Jul 09 13:32:07.813431 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:07.813269 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="kube-rbac-proxy-thanos" containerID="cri-o://755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967" gracePeriod=600 Jul 09 13:32:07.813431 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:07.813334 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="config-reloader" containerID="cri-o://678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3" gracePeriod=600 Jul 09 13:32:07.813431 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:07.813271 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="kube-rbac-proxy-web" containerID="cri-o://fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c" gracePeriod=600 Jul 09 13:32:07.850902 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:07.850851 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.836167213 podStartE2EDuration="2.850829366s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.662153ms" isStatefulPod=false podCreationTimestamp="2026-07-09 13:32:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:32:07.848844329 +0000 UTC m=+145.354146680" watchObservedRunningTime="2026-07-09 13:32:07.850829366 +0000 UTC m=+145.356131657" Jul 09 13:32:08.080343 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.080316 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:08.135521 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.135487 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " Jul 09 13:32:08.135770 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.135538 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-metrics-client-certs\") pod \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " Jul 09 13:32:08.135770 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.135577 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-prometheus-k8s-tls\") pod \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " Jul 09 13:32:08.135770 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.135612 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-thanos-prometheus-http-client-file\") pod \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " Jul 09 13:32:08.135770 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.135670 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-kube-rbac-proxy\") pod \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " Jul 09 13:32:08.135770 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.135698 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/413c5fe9-a023-4ec7-9e45-6141b411d1a1-tls-assets\") pod \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " Jul 09 13:32:08.135770 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.135727 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-grpc-tls\") pod \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " Jul 09 13:32:08.135770 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.135759 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-k8s-db\") pod \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " Jul 09 13:32:08.136147 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.135790 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-trusted-ca-bundle\") pod \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " Jul 09 13:32:08.136147 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.135829 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-k8s-rulefiles-0\") pod \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " Jul 09 13:32:08.136147 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.135868 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-k8s-rulefiles-2\") pod \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " Jul 09 13:32:08.136147 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.135902 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-configmap-kubelet-serving-ca-bundle\") pod \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " Jul 09 13:32:08.136147 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.135954 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-config\") pod \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " Jul 09 13:32:08.136147 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.135986 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/413c5fe9-a023-4ec7-9e45-6141b411d1a1-config-out\") pod \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " Jul 09 13:32:08.136147 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.136040 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " Jul 09 13:32:08.136147 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.136068 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-web-config\") pod \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " Jul 09 13:32:08.136147 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.136093 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-k8s-rulefiles-1\") pod \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " Jul 09 13:32:08.136147 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.136134 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-configmap-serving-certs-ca-bundle\") pod \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " Jul 09 13:32:08.136600 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.136160 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c8ttr\" (UniqueName: \"kubernetes.io/projected/413c5fe9-a023-4ec7-9e45-6141b411d1a1-kube-api-access-c8ttr\") pod \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " Jul 09 13:32:08.136600 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.136186 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-configmap-metrics-client-ca\") pod \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\" (UID: \"413c5fe9-a023-4ec7-9e45-6141b411d1a1\") " Jul 09 13:32:08.136600 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.136358 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "413c5fe9-a023-4ec7-9e45-6141b411d1a1" (UID: "413c5fe9-a023-4ec7-9e45-6141b411d1a1"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:32:08.136600 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.136546 2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:08.137112 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.136839 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "413c5fe9-a023-4ec7-9e45-6141b411d1a1" (UID: "413c5fe9-a023-4ec7-9e45-6141b411d1a1"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:32:08.137112 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.136967 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "413c5fe9-a023-4ec7-9e45-6141b411d1a1" (UID: "413c5fe9-a023-4ec7-9e45-6141b411d1a1"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:32:08.137393 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.137364 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "413c5fe9-a023-4ec7-9e45-6141b411d1a1" (UID: "413c5fe9-a023-4ec7-9e45-6141b411d1a1"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:32:08.137629 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.137606 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "413c5fe9-a023-4ec7-9e45-6141b411d1a1" (UID: "413c5fe9-a023-4ec7-9e45-6141b411d1a1"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:32:08.138865 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.138836 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "413c5fe9-a023-4ec7-9e45-6141b411d1a1" (UID: "413c5fe9-a023-4ec7-9e45-6141b411d1a1"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:32:08.140154 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.139989 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "413c5fe9-a023-4ec7-9e45-6141b411d1a1" (UID: "413c5fe9-a023-4ec7-9e45-6141b411d1a1"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:32:08.140154 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.139993 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "413c5fe9-a023-4ec7-9e45-6141b411d1a1" (UID: "413c5fe9-a023-4ec7-9e45-6141b411d1a1"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:32:08.140546 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.140485 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "413c5fe9-a023-4ec7-9e45-6141b411d1a1" (UID: "413c5fe9-a023-4ec7-9e45-6141b411d1a1"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:32:08.143085 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.140629 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "413c5fe9-a023-4ec7-9e45-6141b411d1a1" (UID: "413c5fe9-a023-4ec7-9e45-6141b411d1a1"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:32:08.145661 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.145424 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/413c5fe9-a023-4ec7-9e45-6141b411d1a1-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "413c5fe9-a023-4ec7-9e45-6141b411d1a1" (UID: "413c5fe9-a023-4ec7-9e45-6141b411d1a1"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 13:32:08.145661 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.145514 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "413c5fe9-a023-4ec7-9e45-6141b411d1a1" (UID: "413c5fe9-a023-4ec7-9e45-6141b411d1a1"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:32:08.145989 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.145959 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/413c5fe9-a023-4ec7-9e45-6141b411d1a1-kube-api-access-c8ttr" (OuterVolumeSpecName: "kube-api-access-c8ttr") pod "413c5fe9-a023-4ec7-9e45-6141b411d1a1" (UID: "413c5fe9-a023-4ec7-9e45-6141b411d1a1"). InnerVolumeSpecName "kube-api-access-c8ttr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 13:32:08.146392 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.146160 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "413c5fe9-a023-4ec7-9e45-6141b411d1a1" (UID: "413c5fe9-a023-4ec7-9e45-6141b411d1a1"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:32:08.146392 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.146228 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-config" (OuterVolumeSpecName: "config") pod "413c5fe9-a023-4ec7-9e45-6141b411d1a1" (UID: "413c5fe9-a023-4ec7-9e45-6141b411d1a1"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:32:08.146392 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.146336 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/413c5fe9-a023-4ec7-9e45-6141b411d1a1-config-out" (OuterVolumeSpecName: "config-out") pod "413c5fe9-a023-4ec7-9e45-6141b411d1a1" (UID: "413c5fe9-a023-4ec7-9e45-6141b411d1a1"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:32:08.146558 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.146408 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "413c5fe9-a023-4ec7-9e45-6141b411d1a1" (UID: "413c5fe9-a023-4ec7-9e45-6141b411d1a1"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:32:08.146558 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.146530 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "413c5fe9-a023-4ec7-9e45-6141b411d1a1" (UID: "413c5fe9-a023-4ec7-9e45-6141b411d1a1"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:32:08.146681 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.146580 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "413c5fe9-a023-4ec7-9e45-6141b411d1a1" (UID: "413c5fe9-a023-4ec7-9e45-6141b411d1a1"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:32:08.156756 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.156717 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-web-config" (OuterVolumeSpecName: "web-config") pod "413c5fe9-a023-4ec7-9e45-6141b411d1a1" (UID: "413c5fe9-a023-4ec7-9e45-6141b411d1a1"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:32:08.237335 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.237296 2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:08.237335 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.237326 2573 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:08.237335 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.237337 2573 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-config\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:08.237335 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.237351 2573 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/413c5fe9-a023-4ec7-9e45-6141b411d1a1-config-out\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:08.237600 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.237360 2573 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:08.237600 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.237370 2573 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-web-config\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:08.237600 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.237378 2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:08.237600 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.237387 2573 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:08.237600 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.237396 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8ttr\" (UniqueName: \"kubernetes.io/projected/413c5fe9-a023-4ec7-9e45-6141b411d1a1-kube-api-access-c8ttr\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:08.237600 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.237404 2573 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-configmap-metrics-client-ca\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:08.237600 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.237412 2573 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:08.237600 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.237421 2573 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-metrics-client-certs\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:08.237600 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.237429 2573 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-prometheus-k8s-tls\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:08.237600 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.237437 2573 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-thanos-prometheus-http-client-file\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:08.237600 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.237446 2573 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-kube-rbac-proxy\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:08.237600 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.237454 2573 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/413c5fe9-a023-4ec7-9e45-6141b411d1a1-tls-assets\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:08.237600 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.237462 2573 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/413c5fe9-a023-4ec7-9e45-6141b411d1a1-secret-grpc-tls\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:08.237600 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.237469 2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-k8s-db\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:08.237600 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.237477 2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/413c5fe9-a023-4ec7-9e45-6141b411d1a1-prometheus-trusted-ca-bundle\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:08.820408 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.820373 2573 generic.go:356] "Generic (PLEG): container finished" podID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerID="755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967" exitCode=0 Jul 09 13:32:08.820408 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.820399 2573 generic.go:356] "Generic (PLEG): container finished" podID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerID="d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae" exitCode=0 Jul 09 13:32:08.820408 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.820405 2573 generic.go:356] "Generic (PLEG): container finished" podID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerID="fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c" exitCode=0 Jul 09 13:32:08.820408 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.820410 2573 generic.go:356] "Generic (PLEG): container finished" podID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerID="abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784" exitCode=0 Jul 09 13:32:08.820408 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.820415 2573 generic.go:356] "Generic (PLEG): container finished" podID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerID="678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3" exitCode=0 Jul 09 13:32:08.820408 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.820421 2573 generic.go:356] "Generic (PLEG): container finished" podID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerID="b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c" exitCode=0 Jul 09 13:32:08.821037 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.820467 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:08.821037 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.820468 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"413c5fe9-a023-4ec7-9e45-6141b411d1a1","Type":"ContainerDied","Data":"755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967"} Jul 09 13:32:08.821037 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.820617 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"413c5fe9-a023-4ec7-9e45-6141b411d1a1","Type":"ContainerDied","Data":"d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae"} Jul 09 13:32:08.821037 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.820663 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"413c5fe9-a023-4ec7-9e45-6141b411d1a1","Type":"ContainerDied","Data":"fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c"} Jul 09 13:32:08.821037 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.820685 2573 scope.go:117] "RemoveContainer" containerID="755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967" Jul 09 13:32:08.821037 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.820705 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"413c5fe9-a023-4ec7-9e45-6141b411d1a1","Type":"ContainerDied","Data":"abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784"} Jul 09 13:32:08.821037 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.820748 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"413c5fe9-a023-4ec7-9e45-6141b411d1a1","Type":"ContainerDied","Data":"678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3"} Jul 09 13:32:08.821037 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.820791 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"413c5fe9-a023-4ec7-9e45-6141b411d1a1","Type":"ContainerDied","Data":"b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c"} Jul 09 13:32:08.821037 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.820831 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"413c5fe9-a023-4ec7-9e45-6141b411d1a1","Type":"ContainerDied","Data":"d9e61cfad5c0de7d250ab81073b854c3ea3b1de4c2c26c38d337e601994da271"} Jul 09 13:32:08.829241 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.829216 2573 scope.go:117] "RemoveContainer" containerID="d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae" Jul 09 13:32:08.841686 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.841536 2573 scope.go:117] "RemoveContainer" containerID="fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c" Jul 09 13:32:08.851646 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.851615 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 13:32:08.855773 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.855746 2573 scope.go:117] "RemoveContainer" containerID="abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784" Jul 09 13:32:08.858042 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.858002 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 13:32:08.863860 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.863837 2573 scope.go:117] "RemoveContainer" containerID="678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3" Jul 09 13:32:08.871356 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.871336 2573 scope.go:117] "RemoveContainer" containerID="b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c" Jul 09 13:32:08.879163 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.879141 2573 scope.go:117] "RemoveContainer" containerID="9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c" Jul 09 13:32:08.887073 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.887051 2573 scope.go:117] "RemoveContainer" containerID="755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967" Jul 09 13:32:08.887404 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:32:08.887379 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967\": container with ID starting with 755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967 not found: ID does not exist" containerID="755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967" Jul 09 13:32:08.887518 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.887412 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967"} err="failed to get container status \"755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967\": rpc error: code = NotFound desc = could not find container \"755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967\": container with ID starting with 755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967 not found: ID does not exist" Jul 09 13:32:08.887518 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.887432 2573 scope.go:117] "RemoveContainer" containerID="d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae" Jul 09 13:32:08.887702 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:32:08.887687 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae\": container with ID starting with d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae not found: ID does not exist" containerID="d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae" Jul 09 13:32:08.887744 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.887708 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae"} err="failed to get container status \"d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae\": rpc error: code = NotFound desc = could not find container \"d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae\": container with ID starting with d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae not found: ID does not exist" Jul 09 13:32:08.887744 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.887724 2573 scope.go:117] "RemoveContainer" containerID="fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c" Jul 09 13:32:08.887972 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:32:08.887955 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c\": container with ID starting with fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c not found: ID does not exist" containerID="fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c" Jul 09 13:32:08.888010 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.887978 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c"} err="failed to get container status \"fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c\": rpc error: code = NotFound desc = could not find container \"fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c\": container with ID starting with fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c not found: ID does not exist" Jul 09 13:32:08.888010 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.887994 2573 scope.go:117] "RemoveContainer" containerID="abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784" Jul 09 13:32:08.888333 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:32:08.888314 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784\": container with ID starting with abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784 not found: ID does not exist" containerID="abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784" Jul 09 13:32:08.888390 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.888335 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784"} err="failed to get container status \"abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784\": rpc error: code = NotFound desc = could not find container \"abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784\": container with ID starting with abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784 not found: ID does not exist" Jul 09 13:32:08.888390 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.888351 2573 scope.go:117] "RemoveContainer" containerID="678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3" Jul 09 13:32:08.888590 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:32:08.888576 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3\": container with ID starting with 678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3 not found: ID does not exist" containerID="678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3" Jul 09 13:32:08.888633 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.888593 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3"} err="failed to get container status \"678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3\": rpc error: code = NotFound desc = could not find container \"678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3\": container with ID starting with 678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3 not found: ID does not exist" Jul 09 13:32:08.888633 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.888610 2573 scope.go:117] "RemoveContainer" containerID="b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c" Jul 09 13:32:08.888848 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:32:08.888832 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c\": container with ID starting with b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c not found: ID does not exist" containerID="b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c" Jul 09 13:32:08.888888 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.888851 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c"} err="failed to get container status \"b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c\": rpc error: code = NotFound desc = could not find container \"b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c\": container with ID starting with b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c not found: ID does not exist" Jul 09 13:32:08.888888 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.888865 2573 scope.go:117] "RemoveContainer" containerID="9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c" Jul 09 13:32:08.889108 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:32:08.889088 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c\": container with ID starting with 9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c not found: ID does not exist" containerID="9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c" Jul 09 13:32:08.889193 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.889109 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c"} err="failed to get container status \"9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c\": rpc error: code = NotFound desc = could not find container \"9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c\": container with ID starting with 9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c not found: ID does not exist" Jul 09 13:32:08.889193 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.889124 2573 scope.go:117] "RemoveContainer" containerID="755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967" Jul 09 13:32:08.889407 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.889388 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967"} err="failed to get container status \"755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967\": rpc error: code = NotFound desc = could not find container \"755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967\": container with ID starting with 755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967 not found: ID does not exist" Jul 09 13:32:08.889449 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.889408 2573 scope.go:117] "RemoveContainer" containerID="d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae" Jul 09 13:32:08.889627 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.889609 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae"} err="failed to get container status \"d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae\": rpc error: code = NotFound desc = could not find container \"d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae\": container with ID starting with d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae not found: ID does not exist" Jul 09 13:32:08.889627 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.889627 2573 scope.go:117] "RemoveContainer" containerID="fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c" Jul 09 13:32:08.889816 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.889797 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c"} err="failed to get container status \"fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c\": rpc error: code = NotFound desc = could not find container \"fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c\": container with ID starting with fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c not found: ID does not exist" Jul 09 13:32:08.889861 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.889817 2573 scope.go:117] "RemoveContainer" containerID="abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784" Jul 09 13:32:08.890042 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.890005 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784"} err="failed to get container status \"abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784\": rpc error: code = NotFound desc = could not find container \"abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784\": container with ID starting with abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784 not found: ID does not exist" Jul 09 13:32:08.890042 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.890040 2573 scope.go:117] "RemoveContainer" containerID="678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3" Jul 09 13:32:08.890248 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.890224 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3"} err="failed to get container status \"678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3\": rpc error: code = NotFound desc = could not find container \"678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3\": container with ID starting with 678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3 not found: ID does not exist" Jul 09 13:32:08.890301 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.890250 2573 scope.go:117] "RemoveContainer" containerID="b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c" Jul 09 13:32:08.890465 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.890442 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c"} err="failed to get container status \"b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c\": rpc error: code = NotFound desc = could not find container \"b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c\": container with ID starting with b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c not found: ID does not exist" Jul 09 13:32:08.890535 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.890468 2573 scope.go:117] "RemoveContainer" containerID="9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c" Jul 09 13:32:08.890700 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.890682 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c"} err="failed to get container status \"9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c\": rpc error: code = NotFound desc = could not find container \"9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c\": container with ID starting with 9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c not found: ID does not exist" Jul 09 13:32:08.890748 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.890700 2573 scope.go:117] "RemoveContainer" containerID="755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967" Jul 09 13:32:08.890972 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.890948 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967"} err="failed to get container status \"755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967\": rpc error: code = NotFound desc = could not find container \"755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967\": container with ID starting with 755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967 not found: ID does not exist" Jul 09 13:32:08.891076 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.890972 2573 scope.go:117] "RemoveContainer" containerID="d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae" Jul 09 13:32:08.891235 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.891217 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae"} err="failed to get container status \"d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae\": rpc error: code = NotFound desc = could not find container \"d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae\": container with ID starting with d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae not found: ID does not exist" Jul 09 13:32:08.891286 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.891236 2573 scope.go:117] "RemoveContainer" containerID="fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c" Jul 09 13:32:08.892103 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.891897 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c"} err="failed to get container status \"fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c\": rpc error: code = NotFound desc = could not find container \"fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c\": container with ID starting with fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c not found: ID does not exist" Jul 09 13:32:08.892103 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.891973 2573 scope.go:117] "RemoveContainer" containerID="abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784" Jul 09 13:32:08.892381 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.892339 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784"} err="failed to get container status \"abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784\": rpc error: code = NotFound desc = could not find container \"abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784\": container with ID starting with abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784 not found: ID does not exist" Jul 09 13:32:08.892381 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.892364 2573 scope.go:117] "RemoveContainer" containerID="678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3" Jul 09 13:32:08.892630 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.892610 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3"} err="failed to get container status \"678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3\": rpc error: code = NotFound desc = could not find container \"678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3\": container with ID starting with 678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3 not found: ID does not exist" Jul 09 13:32:08.892683 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.892631 2573 scope.go:117] "RemoveContainer" containerID="b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c" Jul 09 13:32:08.892880 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.892859 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c"} err="failed to get container status \"b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c\": rpc error: code = NotFound desc = could not find container \"b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c\": container with ID starting with b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c not found: ID does not exist" Jul 09 13:32:08.892933 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.892883 2573 scope.go:117] "RemoveContainer" containerID="9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c" Jul 09 13:32:08.893143 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.893126 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c"} err="failed to get container status \"9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c\": rpc error: code = NotFound desc = could not find container \"9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c\": container with ID starting with 9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c not found: ID does not exist" Jul 09 13:32:08.893143 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.893142 2573 scope.go:117] "RemoveContainer" containerID="755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967" Jul 09 13:32:08.893333 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.893314 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967"} err="failed to get container status \"755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967\": rpc error: code = NotFound desc = could not find container \"755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967\": container with ID starting with 755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967 not found: ID does not exist" Jul 09 13:32:08.893397 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.893336 2573 scope.go:117] "RemoveContainer" containerID="d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae" Jul 09 13:32:08.893579 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.893561 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae"} err="failed to get container status \"d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae\": rpc error: code = NotFound desc = could not find container \"d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae\": container with ID starting with d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae not found: ID does not exist" Jul 09 13:32:08.893638 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.893580 2573 scope.go:117] "RemoveContainer" containerID="fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c" Jul 09 13:32:08.893809 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.893787 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c"} err="failed to get container status \"fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c\": rpc error: code = NotFound desc = could not find container \"fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c\": container with ID starting with fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c not found: ID does not exist" Jul 09 13:32:08.893857 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.893812 2573 scope.go:117] "RemoveContainer" containerID="abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784" Jul 09 13:32:08.894227 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894205 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784"} err="failed to get container status \"abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784\": rpc error: code = NotFound desc = could not find container \"abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784\": container with ID starting with abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784 not found: ID does not exist" Jul 09 13:32:08.894227 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894219 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 13:32:08.894227 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894228 2573 scope.go:117] "RemoveContainer" containerID="678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3" Jul 09 13:32:08.894529 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894511 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3"} err="failed to get container status \"678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3\": rpc error: code = NotFound desc = could not find container \"678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3\": container with ID starting with 678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3 not found: ID does not exist" Jul 09 13:32:08.894579 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894530 2573 scope.go:117] "RemoveContainer" containerID="b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c" Jul 09 13:32:08.894630 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894613 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="kube-rbac-proxy-web" Jul 09 13:32:08.894664 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894635 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="kube-rbac-proxy-web" Jul 09 13:32:08.894664 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894646 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="config-reloader" Jul 09 13:32:08.894664 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894655 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="config-reloader" Jul 09 13:32:08.894801 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894665 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="kube-rbac-proxy" Jul 09 13:32:08.894801 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894674 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="kube-rbac-proxy" Jul 09 13:32:08.894801 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894697 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="prometheus" Jul 09 13:32:08.894801 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894706 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="prometheus" Jul 09 13:32:08.894801 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894721 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="init-config-reloader" Jul 09 13:32:08.894801 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894727 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="init-config-reloader" Jul 09 13:32:08.894801 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894748 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="thanos-sidecar" Jul 09 13:32:08.894801 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894753 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="thanos-sidecar" Jul 09 13:32:08.894801 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894774 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="kube-rbac-proxy-thanos" Jul 09 13:32:08.894801 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894780 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="kube-rbac-proxy-thanos" Jul 09 13:32:08.895178 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894746 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c"} err="failed to get container status \"b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c\": rpc error: code = NotFound desc = could not find container \"b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c\": container with ID starting with b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c not found: ID does not exist" Jul 09 13:32:08.895178 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894846 2573 scope.go:117] "RemoveContainer" containerID="9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c" Jul 09 13:32:08.895178 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894849 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="prometheus" Jul 09 13:32:08.895178 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894905 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="config-reloader" Jul 09 13:32:08.895178 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894915 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="thanos-sidecar" Jul 09 13:32:08.895178 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894924 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="kube-rbac-proxy-web" Jul 09 13:32:08.895178 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894935 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="kube-rbac-proxy-thanos" Jul 09 13:32:08.895178 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.894945 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" containerName="kube-rbac-proxy" Jul 09 13:32:08.895178 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.895095 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c"} err="failed to get container status \"9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c\": rpc error: code = NotFound desc = could not find container \"9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c\": container with ID starting with 9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c not found: ID does not exist" Jul 09 13:32:08.895178 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.895119 2573 scope.go:117] "RemoveContainer" containerID="755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967" Jul 09 13:32:08.895566 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.895364 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967"} err="failed to get container status \"755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967\": rpc error: code = NotFound desc = could not find container \"755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967\": container with ID starting with 755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967 not found: ID does not exist" Jul 09 13:32:08.895566 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.895381 2573 scope.go:117] "RemoveContainer" containerID="d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae" Jul 09 13:32:08.895653 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.895627 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae"} err="failed to get container status \"d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae\": rpc error: code = NotFound desc = could not find container \"d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae\": container with ID starting with d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae not found: ID does not exist" Jul 09 13:32:08.895653 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.895643 2573 scope.go:117] "RemoveContainer" containerID="fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c" Jul 09 13:32:08.895907 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.895887 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c"} err="failed to get container status \"fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c\": rpc error: code = NotFound desc = could not find container \"fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c\": container with ID starting with fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c not found: ID does not exist" Jul 09 13:32:08.895907 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.895907 2573 scope.go:117] "RemoveContainer" containerID="abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784" Jul 09 13:32:08.896152 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.896131 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784"} err="failed to get container status \"abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784\": rpc error: code = NotFound desc = could not find container \"abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784\": container with ID starting with abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784 not found: ID does not exist" Jul 09 13:32:08.896152 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.896152 2573 scope.go:117] "RemoveContainer" containerID="678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3" Jul 09 13:32:08.896400 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.896380 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3"} err="failed to get container status \"678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3\": rpc error: code = NotFound desc = could not find container \"678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3\": container with ID starting with 678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3 not found: ID does not exist" Jul 09 13:32:08.896400 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.896400 2573 scope.go:117] "RemoveContainer" containerID="b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c" Jul 09 13:32:08.896754 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.896727 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c"} err="failed to get container status \"b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c\": rpc error: code = NotFound desc = could not find container \"b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c\": container with ID starting with b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c not found: ID does not exist" Jul 09 13:32:08.896754 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.896754 2573 scope.go:117] "RemoveContainer" containerID="9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c" Jul 09 13:32:08.897037 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.897003 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c"} err="failed to get container status \"9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c\": rpc error: code = NotFound desc = could not find container \"9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c\": container with ID starting with 9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c not found: ID does not exist" Jul 09 13:32:08.897110 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.897039 2573 scope.go:117] "RemoveContainer" containerID="755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967" Jul 09 13:32:08.897288 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.897268 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967"} err="failed to get container status \"755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967\": rpc error: code = NotFound desc = could not find container \"755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967\": container with ID starting with 755d7200e6de14607e44efca2e3ee6e88b342cdc64bc3d1a195b276abd5ad967 not found: ID does not exist" Jul 09 13:32:08.897288 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.897287 2573 scope.go:117] "RemoveContainer" containerID="d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae" Jul 09 13:32:08.897553 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.897531 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae"} err="failed to get container status \"d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae\": rpc error: code = NotFound desc = could not find container \"d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae\": container with ID starting with d94586a7864851e4a35428a3ef045f96af82dfce1cd4e2cb7fe73fc179ffd3ae not found: ID does not exist" Jul 09 13:32:08.897597 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.897553 2573 scope.go:117] "RemoveContainer" containerID="fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c" Jul 09 13:32:08.897774 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.897753 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c"} err="failed to get container status \"fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c\": rpc error: code = NotFound desc = could not find container \"fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c\": container with ID starting with fcdca1e1f2d35523f1f0c9d71aed5ec86296240b745ba3587d0c314da6520f7c not found: ID does not exist" Jul 09 13:32:08.897845 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.897774 2573 scope.go:117] "RemoveContainer" containerID="abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784" Jul 09 13:32:08.898002 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.897981 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784"} err="failed to get container status \"abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784\": rpc error: code = NotFound desc = could not find container \"abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784\": container with ID starting with abcaffbbb658b596ee9989832a46e72f4a01e33acdb2999e801406cc1a880784 not found: ID does not exist" Jul 09 13:32:08.898068 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.898004 2573 scope.go:117] "RemoveContainer" containerID="678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3" Jul 09 13:32:08.898376 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.898347 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3"} err="failed to get container status \"678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3\": rpc error: code = NotFound desc = could not find container \"678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3\": container with ID starting with 678e33a7b933db0712856d6800e197e09160d1b493cc6aede7c32673bdefd8d3 not found: ID does not exist" Jul 09 13:32:08.898457 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.898378 2573 scope.go:117] "RemoveContainer" containerID="b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c" Jul 09 13:32:08.898657 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.898635 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c"} err="failed to get container status \"b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c\": rpc error: code = NotFound desc = could not find container \"b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c\": container with ID starting with b497ef62708055cd55a0eb8e1f27d1c89e83a0e401ab0ffc73671f45937ef42c not found: ID does not exist" Jul 09 13:32:08.898657 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.898656 2573 scope.go:117] "RemoveContainer" containerID="9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c" Jul 09 13:32:08.898964 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.898946 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c"} err="failed to get container status \"9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c\": rpc error: code = NotFound desc = could not find container \"9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c\": container with ID starting with 9d47de617fe039ce98d053e9474e8a89a5d80e9bc5a47c372d736f17e1236a3c not found: ID does not exist" Jul 09 13:32:08.899484 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.899469 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:08.904123 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.904095 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 09 13:32:08.904377 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.904359 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 09 13:32:08.904550 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.904533 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 09 13:32:08.904639 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.904615 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 09 13:32:08.905240 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.905221 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 09 13:32:08.905328 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.905257 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 09 13:32:08.905328 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.905305 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 09 13:32:08.905578 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.905552 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 09 13:32:08.908572 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.908427 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 09 13:32:08.908572 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.908439 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 09 13:32:08.908572 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.908477 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 09 13:32:08.908787 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.908728 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 09 13:32:08.908787 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.908772 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 09 13:32:08.908880 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.908843 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ht5xk\"" Jul 09 13:32:08.909156 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.909130 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9octbmd59c86i\"" Jul 09 13:32:08.914935 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.914907 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 09 13:32:08.917434 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.917405 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 13:32:08.943446 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.943417 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:08.943446 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.943448 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95drx\" (UniqueName: \"kubernetes.io/projected/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-kube-api-access-95drx\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:08.943770 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.943743 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:08.943835 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.943814 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:08.943874 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.943855 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:08.943916 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.943905 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:08.943956 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.943946 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:08.943998 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.943972 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-config\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:08.944068 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.943993 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:08.944142 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.944123 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:08.944256 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.944230 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-config-out\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:08.944367 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.944348 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:08.944413 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.944383 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:08.944450 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.944428 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:08.944503 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.944483 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-web-config\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:08.944587 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.944569 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:08.944655 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.944638 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:08.944719 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.944704 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:08.944833 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.944816 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:08.944877 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.944856 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:08.947609 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:08.947585 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="413c5fe9-a023-4ec7-9e45-6141b411d1a1" path="/var/lib/kubelet/pods/413c5fe9-a023-4ec7-9e45-6141b411d1a1/volumes" Jul 09 13:32:09.045906 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.045865 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.046148 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.045922 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.046148 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.045946 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.046148 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.045963 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.046148 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.045989 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.046148 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.046014 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95drx\" (UniqueName: \"kubernetes.io/projected/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-kube-api-access-95drx\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.046148 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.046091 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.046148 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.046118 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.046526 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.046254 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.046526 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.046300 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.046526 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.046346 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.046526 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.046378 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-config\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.046526 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.046404 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.046526 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.046491 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.046526 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.046519 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-config-out\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.046870 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.046545 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.046870 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.046576 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.046870 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.046613 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.046870 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.046640 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-web-config\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.046870 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.046661 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.046870 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.046670 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.046870 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.046725 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.047258 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.046991 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.047258 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.047137 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.048149 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.048119 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.048367 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.048343 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.048787 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.048761 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.049738 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.049180 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.049738 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.049318 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.051147 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.050794 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-config-out\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.051147 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.050986 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.051147 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.051081 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.051369 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.051224 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.051426 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.051386 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.051426 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.051388 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.051843 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.051827 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-config\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.052088 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.052070 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.052336 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.052315 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.052832 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.052814 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-web-config\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.056505 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.056481 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95drx\" (UniqueName: \"kubernetes.io/projected/1cb1addd-9bb5-44eb-be34-0840e9f7dbfc-kube-api-access-95drx\") pod \"prometheus-k8s-0\" (UID: \"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.210232 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.210199 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:09.362216 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.362188 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 13:32:09.364506 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:32:09.364479 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cb1addd_9bb5_44eb_be34_0840e9f7dbfc.slice/crio-cd9507ba737f116136546a43d9477ccafe8f21607ea491cd7672194063d6efde WatchSource:0}: Error finding container cd9507ba737f116136546a43d9477ccafe8f21607ea491cd7672194063d6efde: Status 404 returned error can't find the container with id cd9507ba737f116136546a43d9477ccafe8f21607ea491cd7672194063d6efde Jul 09 13:32:09.826727 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.826691 2573 generic.go:356] "Generic (PLEG): container finished" podID="1cb1addd-9bb5-44eb-be34-0840e9f7dbfc" containerID="18b219ca0d68fadb82544b6e12ef2effaf550fe4711e6537cefc0c1bc2c1082c" exitCode=0 Jul 09 13:32:09.827235 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.826782 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc","Type":"ContainerDied","Data":"18b219ca0d68fadb82544b6e12ef2effaf550fe4711e6537cefc0c1bc2c1082c"} Jul 09 13:32:09.827235 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:09.826881 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc","Type":"ContainerStarted","Data":"cd9507ba737f116136546a43d9477ccafe8f21607ea491cd7672194063d6efde"} Jul 09 13:32:10.836210 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:10.836172 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc","Type":"ContainerStarted","Data":"7be92d18ab890332b9ed7cd9df8f1ca635d160a6419bb29e1770216ce7027595"} Jul 09 13:32:10.836563 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:10.836280 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc","Type":"ContainerStarted","Data":"304b2657e6afefe9206f084b44313c205129558b24d23be356201214fe793fe4"} Jul 09 13:32:10.836563 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:10.836316 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc","Type":"ContainerStarted","Data":"eeb6ed6d3bc2b1447b09b0f7022bc084819b9340b5b80454740d7753af83e917"} Jul 09 13:32:10.836563 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:10.836341 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc","Type":"ContainerStarted","Data":"4216a5b294c842a6913c02be6ad47eb503ec0e04334029505b817e2a087c0e0d"} Jul 09 13:32:10.836563 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:10.836362 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc","Type":"ContainerStarted","Data":"27f33eea1c81d231cee5a10456744fae246866a257930f52daf773c79ee370a6"} Jul 09 13:32:10.836563 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:10.836395 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1cb1addd-9bb5-44eb-be34-0840e9f7dbfc","Type":"ContainerStarted","Data":"40123aa75f32d87d050456555c1c6070febcba7f50596706216d48c011573a17"} Jul 09 13:32:10.872353 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:10.872295 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.852912424 podStartE2EDuration="2.872271129s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.358705ms" isStatefulPod=false podCreationTimestamp="2026-07-09 13:32:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:32:10.869455243 +0000 UTC m=+148.374757543" watchObservedRunningTime="2026-07-09 13:32:10.872271129 +0000 UTC m=+148.377573441" Jul 09 13:32:14.210390 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:14.210353 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:32:15.855459 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:15.855423 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:15.855893 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:15.855491 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:15.860103 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:15.860079 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:16.867856 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:16.867828 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:32:16.922644 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:16.922610 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f7f8d56cf-4nrfj"] Jul 09 13:32:41.945574 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:41.945528 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5f7f8d56cf-4nrfj" podUID="56a9cac0-8d97-4b72-b8c0-b3adf507d495" containerName="console" containerID="cri-o://a1f81ebf2e8121b86409c8313e921cbbb5f8864a513c5fe5ec261b37e7fea66a" gracePeriod=15 Jul 09 13:32:42.188534 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.188505 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:32:42.243440 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.243339 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/56a9cac0-8d97-4b72-b8c0-b3adf507d495-service-ca\") pod \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " Jul 09 13:32:42.243440 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.243388 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/56a9cac0-8d97-4b72-b8c0-b3adf507d495-oauth-serving-cert\") pod \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " Jul 09 13:32:42.243440 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.243423 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/56a9cac0-8d97-4b72-b8c0-b3adf507d495-console-oauth-config\") pod \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " Jul 09 13:32:42.244069 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.243445 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/56a9cac0-8d97-4b72-b8c0-b3adf507d495-console-config\") pod \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " Jul 09 13:32:42.244069 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.243482 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/56a9cac0-8d97-4b72-b8c0-b3adf507d495-console-serving-cert\") pod \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " Jul 09 13:32:42.244069 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.243517 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74ztf\" (UniqueName: \"kubernetes.io/projected/56a9cac0-8d97-4b72-b8c0-b3adf507d495-kube-api-access-74ztf\") pod \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " Jul 09 13:32:42.244069 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.243547 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56a9cac0-8d97-4b72-b8c0-b3adf507d495-trusted-ca-bundle\") pod \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\" (UID: \"56a9cac0-8d97-4b72-b8c0-b3adf507d495\") " Jul 09 13:32:42.244069 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.243878 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56a9cac0-8d97-4b72-b8c0-b3adf507d495-service-ca" (OuterVolumeSpecName: "service-ca") pod "56a9cac0-8d97-4b72-b8c0-b3adf507d495" (UID: "56a9cac0-8d97-4b72-b8c0-b3adf507d495"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:32:42.244069 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.243888 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56a9cac0-8d97-4b72-b8c0-b3adf507d495-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "56a9cac0-8d97-4b72-b8c0-b3adf507d495" (UID: "56a9cac0-8d97-4b72-b8c0-b3adf507d495"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:32:42.244326 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.244127 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56a9cac0-8d97-4b72-b8c0-b3adf507d495-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "56a9cac0-8d97-4b72-b8c0-b3adf507d495" (UID: "56a9cac0-8d97-4b72-b8c0-b3adf507d495"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:32:42.244326 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.244162 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56a9cac0-8d97-4b72-b8c0-b3adf507d495-console-config" (OuterVolumeSpecName: "console-config") pod "56a9cac0-8d97-4b72-b8c0-b3adf507d495" (UID: "56a9cac0-8d97-4b72-b8c0-b3adf507d495"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:32:42.245975 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.245949 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56a9cac0-8d97-4b72-b8c0-b3adf507d495-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "56a9cac0-8d97-4b72-b8c0-b3adf507d495" (UID: "56a9cac0-8d97-4b72-b8c0-b3adf507d495"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:32:42.246128 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.246066 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56a9cac0-8d97-4b72-b8c0-b3adf507d495-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "56a9cac0-8d97-4b72-b8c0-b3adf507d495" (UID: "56a9cac0-8d97-4b72-b8c0-b3adf507d495"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:32:42.246128 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.246081 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56a9cac0-8d97-4b72-b8c0-b3adf507d495-kube-api-access-74ztf" (OuterVolumeSpecName: "kube-api-access-74ztf") pod "56a9cac0-8d97-4b72-b8c0-b3adf507d495" (UID: "56a9cac0-8d97-4b72-b8c0-b3adf507d495"). InnerVolumeSpecName "kube-api-access-74ztf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 13:32:42.345041 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.344975 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/56a9cac0-8d97-4b72-b8c0-b3adf507d495-oauth-serving-cert\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:42.345041 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.345008 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/56a9cac0-8d97-4b72-b8c0-b3adf507d495-console-oauth-config\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:42.345041 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.345049 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/56a9cac0-8d97-4b72-b8c0-b3adf507d495-console-config\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:42.345287 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.345061 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/56a9cac0-8d97-4b72-b8c0-b3adf507d495-console-serving-cert\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:42.345287 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.345073 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74ztf\" (UniqueName: \"kubernetes.io/projected/56a9cac0-8d97-4b72-b8c0-b3adf507d495-kube-api-access-74ztf\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:42.345287 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.345087 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56a9cac0-8d97-4b72-b8c0-b3adf507d495-trusted-ca-bundle\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:42.345287 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.345098 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/56a9cac0-8d97-4b72-b8c0-b3adf507d495-service-ca\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:32:42.976463 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.976425 2573 generic.go:356] "Generic (PLEG): container finished" podID="56a9cac0-8d97-4b72-b8c0-b3adf507d495" containerID="a1f81ebf2e8121b86409c8313e921cbbb5f8864a513c5fe5ec261b37e7fea66a" exitCode=0 Jul 09 13:32:42.976931 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.976474 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f7f8d56cf-4nrfj" Jul 09 13:32:42.976931 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.976510 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f7f8d56cf-4nrfj" event={"ID":"56a9cac0-8d97-4b72-b8c0-b3adf507d495","Type":"ContainerDied","Data":"a1f81ebf2e8121b86409c8313e921cbbb5f8864a513c5fe5ec261b37e7fea66a"} Jul 09 13:32:42.976931 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.976563 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f7f8d56cf-4nrfj" event={"ID":"56a9cac0-8d97-4b72-b8c0-b3adf507d495","Type":"ContainerDied","Data":"842ec12bb2518dcdab607b8a40da0d4a2351645044c1f00ed8f9cf84fac3b5e6"} Jul 09 13:32:42.976931 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.976587 2573 scope.go:117] "RemoveContainer" containerID="a1f81ebf2e8121b86409c8313e921cbbb5f8864a513c5fe5ec261b37e7fea66a" Jul 09 13:32:42.985070 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.985050 2573 scope.go:117] "RemoveContainer" containerID="a1f81ebf2e8121b86409c8313e921cbbb5f8864a513c5fe5ec261b37e7fea66a" Jul 09 13:32:42.985376 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:32:42.985356 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1f81ebf2e8121b86409c8313e921cbbb5f8864a513c5fe5ec261b37e7fea66a\": container with ID starting with a1f81ebf2e8121b86409c8313e921cbbb5f8864a513c5fe5ec261b37e7fea66a not found: ID does not exist" containerID="a1f81ebf2e8121b86409c8313e921cbbb5f8864a513c5fe5ec261b37e7fea66a" Jul 09 13:32:42.985456 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.985389 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1f81ebf2e8121b86409c8313e921cbbb5f8864a513c5fe5ec261b37e7fea66a"} err="failed to get container status \"a1f81ebf2e8121b86409c8313e921cbbb5f8864a513c5fe5ec261b37e7fea66a\": rpc error: code = NotFound desc = could not find container \"a1f81ebf2e8121b86409c8313e921cbbb5f8864a513c5fe5ec261b37e7fea66a\": container with ID starting with a1f81ebf2e8121b86409c8313e921cbbb5f8864a513c5fe5ec261b37e7fea66a not found: ID does not exist" Jul 09 13:32:42.995579 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.995552 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f7f8d56cf-4nrfj"] Jul 09 13:32:42.999105 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:42.999080 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f7f8d56cf-4nrfj"] Jul 09 13:32:44.945619 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:32:44.945581 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56a9cac0-8d97-4b72-b8c0-b3adf507d495" path="/var/lib/kubelet/pods/56a9cac0-8d97-4b72-b8c0-b3adf507d495/volumes" Jul 09 13:33:09.211345 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:09.211305 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:33:09.227343 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:09.227308 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:33:10.106637 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:10.106608 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 13:33:21.478480 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:21.478446 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-j5lg8"] Jul 09 13:33:21.478981 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:21.478777 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56a9cac0-8d97-4b72-b8c0-b3adf507d495" containerName="console" Jul 09 13:33:21.478981 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:21.478788 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="56a9cac0-8d97-4b72-b8c0-b3adf507d495" containerName="console" Jul 09 13:33:21.478981 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:21.478854 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="56a9cac0-8d97-4b72-b8c0-b3adf507d495" containerName="console" Jul 09 13:33:21.481904 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:21.481879 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-j5lg8" Jul 09 13:33:21.484848 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:21.484827 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 09 13:33:21.485813 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:21.485792 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-k5z8g\"" Jul 09 13:33:21.485911 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:21.485823 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 09 13:33:21.491857 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:21.491832 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-j5lg8"] Jul 09 13:33:21.591780 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:21.591731 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8lmr\" (UniqueName: \"kubernetes.io/projected/f2d2d8db-db3a-4256-a000-e6366e5dc3ba-kube-api-access-l8lmr\") pod \"cert-manager-webhook-6486ffd997-j5lg8\" (UID: \"f2d2d8db-db3a-4256-a000-e6366e5dc3ba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j5lg8" Jul 09 13:33:21.591780 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:21.591787 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2d2d8db-db3a-4256-a000-e6366e5dc3ba-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-j5lg8\" (UID: \"f2d2d8db-db3a-4256-a000-e6366e5dc3ba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j5lg8" Jul 09 13:33:21.692636 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:21.692597 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8lmr\" (UniqueName: \"kubernetes.io/projected/f2d2d8db-db3a-4256-a000-e6366e5dc3ba-kube-api-access-l8lmr\") pod \"cert-manager-webhook-6486ffd997-j5lg8\" (UID: \"f2d2d8db-db3a-4256-a000-e6366e5dc3ba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j5lg8" Jul 09 13:33:21.692821 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:21.692656 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2d2d8db-db3a-4256-a000-e6366e5dc3ba-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-j5lg8\" (UID: \"f2d2d8db-db3a-4256-a000-e6366e5dc3ba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j5lg8" Jul 09 13:33:21.702370 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:21.702342 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8lmr\" (UniqueName: \"kubernetes.io/projected/f2d2d8db-db3a-4256-a000-e6366e5dc3ba-kube-api-access-l8lmr\") pod \"cert-manager-webhook-6486ffd997-j5lg8\" (UID: \"f2d2d8db-db3a-4256-a000-e6366e5dc3ba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j5lg8" Jul 09 13:33:21.702541 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:21.702385 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2d2d8db-db3a-4256-a000-e6366e5dc3ba-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-j5lg8\" (UID: \"f2d2d8db-db3a-4256-a000-e6366e5dc3ba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j5lg8" Jul 09 13:33:21.800973 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:21.800883 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-j5lg8" Jul 09 13:33:21.938738 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:21.937351 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-j5lg8"] Jul 09 13:33:22.139539 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:22.139503 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-j5lg8" event={"ID":"f2d2d8db-db3a-4256-a000-e6366e5dc3ba","Type":"ContainerStarted","Data":"b7affa0d05e7a4e817b55e3e3a6cffa0f576eee6be5c3e1103b474b9af4e7f5d"} Jul 09 13:33:26.160168 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:26.160132 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-j5lg8" event={"ID":"f2d2d8db-db3a-4256-a000-e6366e5dc3ba","Type":"ContainerStarted","Data":"47b55d889cb9e55c9065913051d56acde7df0067f28df02dd06a482cc8196003"} Jul 09 13:33:26.160564 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:26.160271 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-j5lg8" Jul 09 13:33:26.177818 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:26.177516 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-j5lg8" podStartSLOduration=1.7854862169999999 podStartE2EDuration="5.177494018s" totalImagesPullingTime="3.392007801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:33:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:33:26.177342952 +0000 UTC m=+223.682645223" watchObservedRunningTime="2026-07-09 13:33:26.177494018 +0000 UTC m=+223.682796287" Jul 09 13:33:32.166806 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:32.166768 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-j5lg8" Jul 09 13:33:32.488819 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:32.488734 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m2ctd"] Jul 09 13:33:32.491346 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:32.491327 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m2ctd" Jul 09 13:33:32.493787 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:32.493764 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-wsmpz\"" Jul 09 13:33:32.493787 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:32.493765 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 09 13:33:32.494004 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:32.493856 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 09 13:33:32.501529 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:32.501503 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m2ctd"] Jul 09 13:33:32.582165 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:32.582128 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a30c9b2-3f28-43b2-9d2d-ec88af66be0d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-m2ctd\" (UID: \"4a30c9b2-3f28-43b2-9d2d-ec88af66be0d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m2ctd" Jul 09 13:33:32.582165 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:32.582167 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bvjn\" (UniqueName: \"kubernetes.io/projected/4a30c9b2-3f28-43b2-9d2d-ec88af66be0d-kube-api-access-7bvjn\") pod \"openshift-lws-operator-fd8ccff4c-m2ctd\" (UID: \"4a30c9b2-3f28-43b2-9d2d-ec88af66be0d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m2ctd" Jul 09 13:33:32.682713 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:32.682677 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a30c9b2-3f28-43b2-9d2d-ec88af66be0d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-m2ctd\" (UID: \"4a30c9b2-3f28-43b2-9d2d-ec88af66be0d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m2ctd" Jul 09 13:33:32.682713 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:32.682716 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bvjn\" (UniqueName: \"kubernetes.io/projected/4a30c9b2-3f28-43b2-9d2d-ec88af66be0d-kube-api-access-7bvjn\") pod \"openshift-lws-operator-fd8ccff4c-m2ctd\" (UID: \"4a30c9b2-3f28-43b2-9d2d-ec88af66be0d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m2ctd" Jul 09 13:33:32.683141 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:32.683116 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a30c9b2-3f28-43b2-9d2d-ec88af66be0d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-m2ctd\" (UID: \"4a30c9b2-3f28-43b2-9d2d-ec88af66be0d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m2ctd" Jul 09 13:33:32.694782 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:32.694746 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bvjn\" (UniqueName: \"kubernetes.io/projected/4a30c9b2-3f28-43b2-9d2d-ec88af66be0d-kube-api-access-7bvjn\") pod \"openshift-lws-operator-fd8ccff4c-m2ctd\" (UID: \"4a30c9b2-3f28-43b2-9d2d-ec88af66be0d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m2ctd" Jul 09 13:33:32.800845 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:32.800748 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m2ctd" Jul 09 13:33:32.937560 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:32.937525 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m2ctd"] Jul 09 13:33:32.942593 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:33:32.942563 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a30c9b2_3f28_43b2_9d2d_ec88af66be0d.slice/crio-d7ad966aa455c907f7fd401e09482cbfdbecd48df186c7a6709e73d2d71808fb WatchSource:0}: Error finding container d7ad966aa455c907f7fd401e09482cbfdbecd48df186c7a6709e73d2d71808fb: Status 404 returned error can't find the container with id d7ad966aa455c907f7fd401e09482cbfdbecd48df186c7a6709e73d2d71808fb Jul 09 13:33:33.192690 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:33.192647 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m2ctd" event={"ID":"4a30c9b2-3f28-43b2-9d2d-ec88af66be0d","Type":"ContainerStarted","Data":"d7ad966aa455c907f7fd401e09482cbfdbecd48df186c7a6709e73d2d71808fb"} Jul 09 13:33:36.212277 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:36.212238 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m2ctd" event={"ID":"4a30c9b2-3f28-43b2-9d2d-ec88af66be0d","Type":"ContainerStarted","Data":"a4e5ad208b15266278d4916ac3cb10fe360fadd438dbd8b871db16f5887347ef"} Jul 09 13:33:36.232201 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:36.232149 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m2ctd" podStartSLOduration=1.078306036 podStartE2EDuration="4.232134902s" totalImagesPullingTime="3.153828866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:33:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:33:36.230340887 +0000 UTC m=+233.735643159" watchObservedRunningTime="2026-07-09 13:33:36.232134902 +0000 UTC m=+233.737437170" Jul 09 13:33:51.886910 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:51.886870 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2"] Jul 09 13:33:51.894439 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:51.894408 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2" Jul 09 13:33:51.899427 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:51.899397 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 13:33:51.899427 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:51.899425 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ts6jb\"" Jul 09 13:33:51.899658 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:51.899403 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 13:33:51.899658 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:51.899469 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 13:33:51.899807 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:51.899792 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 13:33:51.904434 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:51.904404 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2"] Jul 09 13:33:51.965875 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:51.965835 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/063431c8-3f4f-47d7-bd90-1eba78202a38-tmp\") pod \"opendatahub-operator-controller-manager-69bb5d6bf5-48wl2\" (UID: \"063431c8-3f4f-47d7-bd90-1eba78202a38\") " pod="opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2" Jul 09 13:33:51.966124 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:51.965901 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/063431c8-3f4f-47d7-bd90-1eba78202a38-webhook-cert\") pod \"opendatahub-operator-controller-manager-69bb5d6bf5-48wl2\" (UID: \"063431c8-3f4f-47d7-bd90-1eba78202a38\") " pod="opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2" Jul 09 13:33:51.966124 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:51.966080 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dl62\" (UniqueName: \"kubernetes.io/projected/063431c8-3f4f-47d7-bd90-1eba78202a38-kube-api-access-8dl62\") pod \"opendatahub-operator-controller-manager-69bb5d6bf5-48wl2\" (UID: \"063431c8-3f4f-47d7-bd90-1eba78202a38\") " pod="opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2" Jul 09 13:33:51.966239 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:51.966140 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/063431c8-3f4f-47d7-bd90-1eba78202a38-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69bb5d6bf5-48wl2\" (UID: \"063431c8-3f4f-47d7-bd90-1eba78202a38\") " pod="opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2" Jul 09 13:33:51.966287 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:51.966238 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/063431c8-3f4f-47d7-bd90-1eba78202a38-manifests\") pod \"opendatahub-operator-controller-manager-69bb5d6bf5-48wl2\" (UID: \"063431c8-3f4f-47d7-bd90-1eba78202a38\") " pod="opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2" Jul 09 13:33:52.066939 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:52.066902 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/063431c8-3f4f-47d7-bd90-1eba78202a38-tmp\") pod \"opendatahub-operator-controller-manager-69bb5d6bf5-48wl2\" (UID: \"063431c8-3f4f-47d7-bd90-1eba78202a38\") " pod="opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2" Jul 09 13:33:52.067150 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:52.066945 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/063431c8-3f4f-47d7-bd90-1eba78202a38-webhook-cert\") pod \"opendatahub-operator-controller-manager-69bb5d6bf5-48wl2\" (UID: \"063431c8-3f4f-47d7-bd90-1eba78202a38\") " pod="opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2" Jul 09 13:33:52.067150 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:52.066977 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dl62\" (UniqueName: \"kubernetes.io/projected/063431c8-3f4f-47d7-bd90-1eba78202a38-kube-api-access-8dl62\") pod \"opendatahub-operator-controller-manager-69bb5d6bf5-48wl2\" (UID: \"063431c8-3f4f-47d7-bd90-1eba78202a38\") " pod="opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2" Jul 09 13:33:52.067150 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:52.067007 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/063431c8-3f4f-47d7-bd90-1eba78202a38-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69bb5d6bf5-48wl2\" (UID: \"063431c8-3f4f-47d7-bd90-1eba78202a38\") " pod="opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2" Jul 09 13:33:52.067150 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:52.067075 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/063431c8-3f4f-47d7-bd90-1eba78202a38-manifests\") pod \"opendatahub-operator-controller-manager-69bb5d6bf5-48wl2\" (UID: \"063431c8-3f4f-47d7-bd90-1eba78202a38\") " pod="opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2" Jul 09 13:33:52.067446 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:52.067420 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/063431c8-3f4f-47d7-bd90-1eba78202a38-manifests\") pod \"opendatahub-operator-controller-manager-69bb5d6bf5-48wl2\" (UID: \"063431c8-3f4f-47d7-bd90-1eba78202a38\") " pod="opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2" Jul 09 13:33:52.069378 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:52.069343 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/063431c8-3f4f-47d7-bd90-1eba78202a38-tmp\") pod \"opendatahub-operator-controller-manager-69bb5d6bf5-48wl2\" (UID: \"063431c8-3f4f-47d7-bd90-1eba78202a38\") " pod="opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2" Jul 09 13:33:52.069698 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:52.069676 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/063431c8-3f4f-47d7-bd90-1eba78202a38-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69bb5d6bf5-48wl2\" (UID: \"063431c8-3f4f-47d7-bd90-1eba78202a38\") " pod="opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2" Jul 09 13:33:52.069747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:52.069708 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/063431c8-3f4f-47d7-bd90-1eba78202a38-webhook-cert\") pod \"opendatahub-operator-controller-manager-69bb5d6bf5-48wl2\" (UID: \"063431c8-3f4f-47d7-bd90-1eba78202a38\") " pod="opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2" Jul 09 13:33:52.077662 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:52.077635 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dl62\" (UniqueName: \"kubernetes.io/projected/063431c8-3f4f-47d7-bd90-1eba78202a38-kube-api-access-8dl62\") pod \"opendatahub-operator-controller-manager-69bb5d6bf5-48wl2\" (UID: \"063431c8-3f4f-47d7-bd90-1eba78202a38\") " pod="opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2" Jul 09 13:33:52.206714 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:52.206613 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2" Jul 09 13:33:52.357836 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:52.357755 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2"] Jul 09 13:33:52.360678 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:33:52.360638 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod063431c8_3f4f_47d7_bd90_1eba78202a38.slice/crio-64cf4d811e00e660cf5ce3a67dd904622b2926f33fd73d40b80317f398d22bc3 WatchSource:0}: Error finding container 64cf4d811e00e660cf5ce3a67dd904622b2926f33fd73d40b80317f398d22bc3: Status 404 returned error can't find the container with id 64cf4d811e00e660cf5ce3a67dd904622b2926f33fd73d40b80317f398d22bc3 Jul 09 13:33:53.291446 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:53.291363 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2" event={"ID":"063431c8-3f4f-47d7-bd90-1eba78202a38","Type":"ContainerStarted","Data":"64cf4d811e00e660cf5ce3a67dd904622b2926f33fd73d40b80317f398d22bc3"} Jul 09 13:33:56.309800 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:56.309762 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2" event={"ID":"063431c8-3f4f-47d7-bd90-1eba78202a38","Type":"ContainerStarted","Data":"5deaa48445b59590de6472c22d4998ec42c066621ea65e2977ca4ab967bb6b84"} Jul 09 13:33:57.316324 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:57.316287 2573 generic.go:356] "Generic (PLEG): container finished" podID="063431c8-3f4f-47d7-bd90-1eba78202a38" containerID="5deaa48445b59590de6472c22d4998ec42c066621ea65e2977ca4ab967bb6b84" exitCode=0 Jul 09 13:33:57.316738 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:57.316377 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2" event={"ID":"063431c8-3f4f-47d7-bd90-1eba78202a38","Type":"ContainerDied","Data":"5deaa48445b59590de6472c22d4998ec42c066621ea65e2977ca4ab967bb6b84"} Jul 09 13:33:58.322904 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:58.322865 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2" event={"ID":"063431c8-3f4f-47d7-bd90-1eba78202a38","Type":"ContainerStarted","Data":"1f8d9ca2dd736b8491da5d35eac76e0dff9dbe66de3451013ab8cd2c00c6dd00"} Jul 09 13:33:58.323428 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:58.322981 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2" Jul 09 13:33:58.351434 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:33:58.351369 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2" podStartSLOduration=2.570684485 podStartE2EDuration="7.351345817s" totalImagesPullingTime="3.741610403s" totalInitContainerRuntime="1.039050929s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:33:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:33:58.348635795 +0000 UTC m=+255.853938094" watchObservedRunningTime="2026-07-09 13:33:58.351345817 +0000 UTC m=+255.856648084" Jul 09 13:34:09.329870 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:09.329792 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-69bb5d6bf5-48wl2" Jul 09 13:34:12.160309 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:12.160269 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7769c579dd-9z6lt"] Jul 09 13:34:12.163275 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:12.163251 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7769c579dd-9z6lt" Jul 09 13:34:12.167674 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:12.167645 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 09 13:34:12.167838 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:12.167703 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 09 13:34:12.167838 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:12.167714 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 09 13:34:12.167838 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:12.167655 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-42xfs\"" Jul 09 13:34:12.179599 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:12.179566 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7769c579dd-9z6lt"] Jul 09 13:34:12.260400 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:12.260363 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f-cert\") pod \"lws-controller-manager-7769c579dd-9z6lt\" (UID: \"fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f\") " pod="openshift-lws-operator/lws-controller-manager-7769c579dd-9z6lt" Jul 09 13:34:12.260592 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:12.260420 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f-metrics-cert\") pod \"lws-controller-manager-7769c579dd-9z6lt\" (UID: \"fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f\") " pod="openshift-lws-operator/lws-controller-manager-7769c579dd-9z6lt" Jul 09 13:34:12.260592 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:12.260467 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f-manager-config\") pod \"lws-controller-manager-7769c579dd-9z6lt\" (UID: \"fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f\") " pod="openshift-lws-operator/lws-controller-manager-7769c579dd-9z6lt" Jul 09 13:34:12.260592 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:12.260500 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pxsw\" (UniqueName: \"kubernetes.io/projected/fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f-kube-api-access-7pxsw\") pod \"lws-controller-manager-7769c579dd-9z6lt\" (UID: \"fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f\") " pod="openshift-lws-operator/lws-controller-manager-7769c579dd-9z6lt" Jul 09 13:34:12.360956 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:12.360918 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f-cert\") pod \"lws-controller-manager-7769c579dd-9z6lt\" (UID: \"fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f\") " pod="openshift-lws-operator/lws-controller-manager-7769c579dd-9z6lt" Jul 09 13:34:12.361161 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:12.360976 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f-metrics-cert\") pod \"lws-controller-manager-7769c579dd-9z6lt\" (UID: \"fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f\") " pod="openshift-lws-operator/lws-controller-manager-7769c579dd-9z6lt" Jul 09 13:34:12.361161 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:12.361004 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f-manager-config\") pod \"lws-controller-manager-7769c579dd-9z6lt\" (UID: \"fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f\") " pod="openshift-lws-operator/lws-controller-manager-7769c579dd-9z6lt" Jul 09 13:34:12.361161 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:12.361052 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pxsw\" (UniqueName: \"kubernetes.io/projected/fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f-kube-api-access-7pxsw\") pod \"lws-controller-manager-7769c579dd-9z6lt\" (UID: \"fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f\") " pod="openshift-lws-operator/lws-controller-manager-7769c579dd-9z6lt" Jul 09 13:34:12.361826 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:12.361796 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f-manager-config\") pod \"lws-controller-manager-7769c579dd-9z6lt\" (UID: \"fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f\") " pod="openshift-lws-operator/lws-controller-manager-7769c579dd-9z6lt" Jul 09 13:34:12.363650 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:12.363613 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f-cert\") pod \"lws-controller-manager-7769c579dd-9z6lt\" (UID: \"fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f\") " pod="openshift-lws-operator/lws-controller-manager-7769c579dd-9z6lt" Jul 09 13:34:12.363779 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:12.363613 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f-metrics-cert\") pod \"lws-controller-manager-7769c579dd-9z6lt\" (UID: \"fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f\") " pod="openshift-lws-operator/lws-controller-manager-7769c579dd-9z6lt" Jul 09 13:34:12.378645 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:12.378615 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pxsw\" (UniqueName: \"kubernetes.io/projected/fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f-kube-api-access-7pxsw\") pod \"lws-controller-manager-7769c579dd-9z6lt\" (UID: \"fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f\") " pod="openshift-lws-operator/lws-controller-manager-7769c579dd-9z6lt" Jul 09 13:34:12.473910 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:12.473811 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7769c579dd-9z6lt" Jul 09 13:34:12.617214 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:12.617188 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7769c579dd-9z6lt"] Jul 09 13:34:13.397258 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:13.397210 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7769c579dd-9z6lt" event={"ID":"fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f","Type":"ContainerStarted","Data":"d3618a5c3083f8f30adef162569c9eaf94e7dd9b867a3d36a2714cebb3d5ff66"} Jul 09 13:34:14.405273 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:14.405172 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7769c579dd-9z6lt" event={"ID":"fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f","Type":"ContainerStarted","Data":"3b63dc1af940a5d10e2a6ec6a9d59635daed39c58d87a6b70318c679a735b1b4"} Jul 09 13:34:14.405647 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:14.405283 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7769c579dd-9z6lt" Jul 09 13:34:14.429512 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:14.429445 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7769c579dd-9z6lt" podStartSLOduration=0.901699081 podStartE2EDuration="2.429423561s" totalImagesPullingTime="1.52772448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:34:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:34:14.428988297 +0000 UTC m=+271.934290565" watchObservedRunningTime="2026-07-09 13:34:14.429423561 +0000 UTC m=+271.934725831" Jul 09 13:34:18.937723 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:18.937684 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2"] Jul 09 13:34:18.940250 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:18.940226 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:18.943912 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:18.943879 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 13:34:18.944217 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:18.944192 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-nxpk2\"" Jul 09 13:34:18.944359 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:18.944275 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 13:34:18.944425 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:18.944376 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 13:34:18.961956 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:18.961922 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2"] Jul 09 13:34:19.025993 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.025958 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dab3f11-b82c-443b-82df-a9e18f3bb094-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.025993 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.025993 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dab3f11-b82c-443b-82df-a9e18f3bb094-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.026247 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.026041 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dab3f11-b82c-443b-82df-a9e18f3bb094-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.026247 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.026122 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dab3f11-b82c-443b-82df-a9e18f3bb094-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.026247 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.026175 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dab3f11-b82c-443b-82df-a9e18f3bb094-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.026247 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.026201 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5hpm\" (UniqueName: \"kubernetes.io/projected/9dab3f11-b82c-443b-82df-a9e18f3bb094-kube-api-access-h5hpm\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.026247 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.026223 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dab3f11-b82c-443b-82df-a9e18f3bb094-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.026415 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.026271 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dab3f11-b82c-443b-82df-a9e18f3bb094-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.026415 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.026304 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dab3f11-b82c-443b-82df-a9e18f3bb094-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.127086 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.127011 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dab3f11-b82c-443b-82df-a9e18f3bb094-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.127267 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.127102 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dab3f11-b82c-443b-82df-a9e18f3bb094-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.127267 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.127122 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dab3f11-b82c-443b-82df-a9e18f3bb094-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.127267 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.127143 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dab3f11-b82c-443b-82df-a9e18f3bb094-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.127267 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.127173 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dab3f11-b82c-443b-82df-a9e18f3bb094-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.127267 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.127230 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dab3f11-b82c-443b-82df-a9e18f3bb094-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.127267 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.127259 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5hpm\" (UniqueName: \"kubernetes.io/projected/9dab3f11-b82c-443b-82df-a9e18f3bb094-kube-api-access-h5hpm\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.127552 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.127286 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dab3f11-b82c-443b-82df-a9e18f3bb094-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.127552 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.127334 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dab3f11-b82c-443b-82df-a9e18f3bb094-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.127655 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.127586 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dab3f11-b82c-443b-82df-a9e18f3bb094-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.127950 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.127923 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dab3f11-b82c-443b-82df-a9e18f3bb094-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.127950 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.127941 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dab3f11-b82c-443b-82df-a9e18f3bb094-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.128119 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.127958 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dab3f11-b82c-443b-82df-a9e18f3bb094-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.128119 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.127977 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dab3f11-b82c-443b-82df-a9e18f3bb094-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.129600 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.129573 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dab3f11-b82c-443b-82df-a9e18f3bb094-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.129691 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.129677 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dab3f11-b82c-443b-82df-a9e18f3bb094-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.138674 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.138645 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dab3f11-b82c-443b-82df-a9e18f3bb094-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.138853 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.138832 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5hpm\" (UniqueName: \"kubernetes.io/projected/9dab3f11-b82c-443b-82df-a9e18f3bb094-kube-api-access-h5hpm\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nlfp2\" (UID: \"9dab3f11-b82c-443b-82df-a9e18f3bb094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.251661 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.251554 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:19.405427 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.405253 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2"] Jul 09 13:34:19.407654 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:34:19.407624 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dab3f11_b82c_443b_82df_a9e18f3bb094.slice/crio-6b1733fe59ea1d4386a65a685e176b65a0f8675635ef2e056e23d9e0e0672602 WatchSource:0}: Error finding container 6b1733fe59ea1d4386a65a685e176b65a0f8675635ef2e056e23d9e0e0672602: Status 404 returned error can't find the container with id 6b1733fe59ea1d4386a65a685e176b65a0f8675635ef2e056e23d9e0e0672602 Jul 09 13:34:19.430035 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:19.429987 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" event={"ID":"9dab3f11-b82c-443b-82df-a9e18f3bb094","Type":"ContainerStarted","Data":"6b1733fe59ea1d4386a65a685e176b65a0f8675635ef2e056e23d9e0e0672602"} Jul 09 13:34:21.226893 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.226857 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l"] Jul 09 13:34:21.235581 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.234462 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" Jul 09 13:34:21.237427 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.237344 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 09 13:34:21.237579 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.237341 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 09 13:34:21.237660 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.237632 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 09 13:34:21.237754 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.237343 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-d54nv\"" Jul 09 13:34:21.247273 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.247229 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l"] Jul 09 13:34:21.350669 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.350633 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe184a24-50aa-4d20-8384-8aa523445d26-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-79t5l\" (UID: \"fe184a24-50aa-4d20-8384-8aa523445d26\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" Jul 09 13:34:21.350669 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.350677 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe184a24-50aa-4d20-8384-8aa523445d26-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-79t5l\" (UID: \"fe184a24-50aa-4d20-8384-8aa523445d26\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" Jul 09 13:34:21.350936 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.350709 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fe184a24-50aa-4d20-8384-8aa523445d26-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-79t5l\" (UID: \"fe184a24-50aa-4d20-8384-8aa523445d26\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" Jul 09 13:34:21.350936 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.350774 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fe184a24-50aa-4d20-8384-8aa523445d26-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-79t5l\" (UID: \"fe184a24-50aa-4d20-8384-8aa523445d26\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" Jul 09 13:34:21.350936 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.350799 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msrjn\" (UniqueName: \"kubernetes.io/projected/fe184a24-50aa-4d20-8384-8aa523445d26-kube-api-access-msrjn\") pod \"istiod-openshift-gateway-94bb8fbfd-79t5l\" (UID: \"fe184a24-50aa-4d20-8384-8aa523445d26\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" Jul 09 13:34:21.350936 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.350844 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fe184a24-50aa-4d20-8384-8aa523445d26-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-79t5l\" (UID: \"fe184a24-50aa-4d20-8384-8aa523445d26\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" Jul 09 13:34:21.350936 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.350862 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fe184a24-50aa-4d20-8384-8aa523445d26-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-79t5l\" (UID: \"fe184a24-50aa-4d20-8384-8aa523445d26\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" Jul 09 13:34:21.451755 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.451707 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe184a24-50aa-4d20-8384-8aa523445d26-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-79t5l\" (UID: \"fe184a24-50aa-4d20-8384-8aa523445d26\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" Jul 09 13:34:21.451755 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.451760 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe184a24-50aa-4d20-8384-8aa523445d26-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-79t5l\" (UID: \"fe184a24-50aa-4d20-8384-8aa523445d26\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" Jul 09 13:34:21.452032 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.451797 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fe184a24-50aa-4d20-8384-8aa523445d26-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-79t5l\" (UID: \"fe184a24-50aa-4d20-8384-8aa523445d26\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" Jul 09 13:34:21.452032 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.451853 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fe184a24-50aa-4d20-8384-8aa523445d26-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-79t5l\" (UID: \"fe184a24-50aa-4d20-8384-8aa523445d26\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" Jul 09 13:34:21.452032 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.451883 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msrjn\" (UniqueName: \"kubernetes.io/projected/fe184a24-50aa-4d20-8384-8aa523445d26-kube-api-access-msrjn\") pod \"istiod-openshift-gateway-94bb8fbfd-79t5l\" (UID: \"fe184a24-50aa-4d20-8384-8aa523445d26\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" Jul 09 13:34:21.452032 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.451926 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fe184a24-50aa-4d20-8384-8aa523445d26-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-79t5l\" (UID: \"fe184a24-50aa-4d20-8384-8aa523445d26\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" Jul 09 13:34:21.452032 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.451953 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fe184a24-50aa-4d20-8384-8aa523445d26-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-79t5l\" (UID: \"fe184a24-50aa-4d20-8384-8aa523445d26\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" Jul 09 13:34:21.452949 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.452897 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fe184a24-50aa-4d20-8384-8aa523445d26-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-79t5l\" (UID: \"fe184a24-50aa-4d20-8384-8aa523445d26\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" Jul 09 13:34:21.454782 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.454753 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fe184a24-50aa-4d20-8384-8aa523445d26-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-79t5l\" (UID: \"fe184a24-50aa-4d20-8384-8aa523445d26\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" Jul 09 13:34:21.454907 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.454785 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe184a24-50aa-4d20-8384-8aa523445d26-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-79t5l\" (UID: \"fe184a24-50aa-4d20-8384-8aa523445d26\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" Jul 09 13:34:21.455001 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.454979 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fe184a24-50aa-4d20-8384-8aa523445d26-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-79t5l\" (UID: \"fe184a24-50aa-4d20-8384-8aa523445d26\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" Jul 09 13:34:21.455084 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.454986 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fe184a24-50aa-4d20-8384-8aa523445d26-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-79t5l\" (UID: \"fe184a24-50aa-4d20-8384-8aa523445d26\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" Jul 09 13:34:21.467275 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.467235 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msrjn\" (UniqueName: \"kubernetes.io/projected/fe184a24-50aa-4d20-8384-8aa523445d26-kube-api-access-msrjn\") pod \"istiod-openshift-gateway-94bb8fbfd-79t5l\" (UID: \"fe184a24-50aa-4d20-8384-8aa523445d26\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" Jul 09 13:34:21.467758 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.467735 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe184a24-50aa-4d20-8384-8aa523445d26-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-79t5l\" (UID: \"fe184a24-50aa-4d20-8384-8aa523445d26\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" Jul 09 13:34:21.552927 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.552801 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" Jul 09 13:34:21.841083 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.840991 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:34:21.841224 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.841116 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:34:21.841224 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.841145 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:34:21.967078 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:21.966413 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l"] Jul 09 13:34:21.971820 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:34:21.971770 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe184a24_50aa_4d20_8384_8aa523445d26.slice/crio-aa46cb9db8cf59c6ec6af0a005aadd417bcd3ade23a3af489564afa8e81bb78f WatchSource:0}: Error finding container aa46cb9db8cf59c6ec6af0a005aadd417bcd3ade23a3af489564afa8e81bb78f: Status 404 returned error can't find the container with id aa46cb9db8cf59c6ec6af0a005aadd417bcd3ade23a3af489564afa8e81bb78f Jul 09 13:34:22.449594 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:22.449551 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" event={"ID":"fe184a24-50aa-4d20-8384-8aa523445d26","Type":"ContainerStarted","Data":"aa46cb9db8cf59c6ec6af0a005aadd417bcd3ade23a3af489564afa8e81bb78f"} Jul 09 13:34:22.451426 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:22.451392 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" event={"ID":"9dab3f11-b82c-443b-82df-a9e18f3bb094","Type":"ContainerStarted","Data":"92e5c81067cc3f17f47f1139b41f452ef856cc97f5c2be627d227a5b04d107d3"} Jul 09 13:34:22.484701 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:22.484644 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" podStartSLOduration=2.053738024 podStartE2EDuration="4.484625283s" totalImagesPullingTime="2.430887259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:34:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:34:22.482529167 +0000 UTC m=+279.987831459" watchObservedRunningTime="2026-07-09 13:34:22.484625283 +0000 UTC m=+279.989927561" Jul 09 13:34:23.251792 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:23.251751 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:23.257704 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:23.257674 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:23.458481 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:23.458444 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:23.459696 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:23.459675 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nlfp2" Jul 09 13:34:24.031419 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:24.031303 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:34:24.031419 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:24.031397 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:34:24.466157 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:24.466041 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" event={"ID":"fe184a24-50aa-4d20-8384-8aa523445d26","Type":"ContainerStarted","Data":"5095dcdde220c839089873c361b3d3043fb3986453e44ea15bf419ef27824f55"} Jul 09 13:34:24.466652 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:24.466357 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" Jul 09 13:34:24.506119 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:24.506054 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" podStartSLOduration=1.449429182 podStartE2EDuration="3.506031358s" totalImagesPullingTime="2.056602176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:34:24.505753301 +0000 UTC m=+282.011055596" watchObservedRunningTime="2026-07-09 13:34:24.506031358 +0000 UTC m=+282.011333829" Jul 09 13:34:25.412741 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:25.412707 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7769c579dd-9z6lt" Jul 09 13:34:25.474036 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:25.473984 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-79t5l" Jul 09 13:34:42.845921 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:42.845886 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmms8_3b512d09-0288-4fea-864e-fdc734afb1bd/ovn-acl-logging/0.log" Jul 09 13:34:42.846341 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:42.846222 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmms8_3b512d09-0288-4fea-864e-fdc734afb1bd/ovn-acl-logging/0.log" Jul 09 13:34:42.849921 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:34:42.849902 2573 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 13:35:58.339819 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.339781 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78c8755b99-kbc6p"] Jul 09 13:35:58.342760 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.342738 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:35:58.355212 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.355184 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78c8755b99-kbc6p"] Jul 09 13:35:58.487308 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.487268 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e2abc06-b17d-40a8-a2e4-8048059b43ad-console-oauth-config\") pod \"console-78c8755b99-kbc6p\" (UID: \"0e2abc06-b17d-40a8-a2e4-8048059b43ad\") " pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:35:58.487308 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.487306 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e2abc06-b17d-40a8-a2e4-8048059b43ad-service-ca\") pod \"console-78c8755b99-kbc6p\" (UID: \"0e2abc06-b17d-40a8-a2e4-8048059b43ad\") " pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:35:58.487540 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.487333 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e2abc06-b17d-40a8-a2e4-8048059b43ad-console-config\") pod \"console-78c8755b99-kbc6p\" (UID: \"0e2abc06-b17d-40a8-a2e4-8048059b43ad\") " pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:35:58.487540 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.487435 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e2abc06-b17d-40a8-a2e4-8048059b43ad-console-serving-cert\") pod \"console-78c8755b99-kbc6p\" (UID: \"0e2abc06-b17d-40a8-a2e4-8048059b43ad\") " pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:35:58.487540 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.487475 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvpfd\" (UniqueName: \"kubernetes.io/projected/0e2abc06-b17d-40a8-a2e4-8048059b43ad-kube-api-access-hvpfd\") pod \"console-78c8755b99-kbc6p\" (UID: \"0e2abc06-b17d-40a8-a2e4-8048059b43ad\") " pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:35:58.487540 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.487496 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e2abc06-b17d-40a8-a2e4-8048059b43ad-oauth-serving-cert\") pod \"console-78c8755b99-kbc6p\" (UID: \"0e2abc06-b17d-40a8-a2e4-8048059b43ad\") " pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:35:58.487677 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.487595 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e2abc06-b17d-40a8-a2e4-8048059b43ad-trusted-ca-bundle\") pod \"console-78c8755b99-kbc6p\" (UID: \"0e2abc06-b17d-40a8-a2e4-8048059b43ad\") " pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:35:58.589014 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.588971 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvpfd\" (UniqueName: \"kubernetes.io/projected/0e2abc06-b17d-40a8-a2e4-8048059b43ad-kube-api-access-hvpfd\") pod \"console-78c8755b99-kbc6p\" (UID: \"0e2abc06-b17d-40a8-a2e4-8048059b43ad\") " pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:35:58.589014 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.589012 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e2abc06-b17d-40a8-a2e4-8048059b43ad-oauth-serving-cert\") pod \"console-78c8755b99-kbc6p\" (UID: \"0e2abc06-b17d-40a8-a2e4-8048059b43ad\") " pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:35:58.589257 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.589088 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e2abc06-b17d-40a8-a2e4-8048059b43ad-trusted-ca-bundle\") pod \"console-78c8755b99-kbc6p\" (UID: \"0e2abc06-b17d-40a8-a2e4-8048059b43ad\") " pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:35:58.589320 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.589298 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e2abc06-b17d-40a8-a2e4-8048059b43ad-console-oauth-config\") pod \"console-78c8755b99-kbc6p\" (UID: \"0e2abc06-b17d-40a8-a2e4-8048059b43ad\") " pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:35:58.589370 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.589333 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e2abc06-b17d-40a8-a2e4-8048059b43ad-service-ca\") pod \"console-78c8755b99-kbc6p\" (UID: \"0e2abc06-b17d-40a8-a2e4-8048059b43ad\") " pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:35:58.589429 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.589368 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e2abc06-b17d-40a8-a2e4-8048059b43ad-console-config\") pod \"console-78c8755b99-kbc6p\" (UID: \"0e2abc06-b17d-40a8-a2e4-8048059b43ad\") " pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:35:58.589429 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.589406 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e2abc06-b17d-40a8-a2e4-8048059b43ad-console-serving-cert\") pod \"console-78c8755b99-kbc6p\" (UID: \"0e2abc06-b17d-40a8-a2e4-8048059b43ad\") " pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:35:58.589968 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.589901 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e2abc06-b17d-40a8-a2e4-8048059b43ad-oauth-serving-cert\") pod \"console-78c8755b99-kbc6p\" (UID: \"0e2abc06-b17d-40a8-a2e4-8048059b43ad\") " pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:35:58.590118 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.590002 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e2abc06-b17d-40a8-a2e4-8048059b43ad-trusted-ca-bundle\") pod \"console-78c8755b99-kbc6p\" (UID: \"0e2abc06-b17d-40a8-a2e4-8048059b43ad\") " pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:35:58.590118 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.590084 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e2abc06-b17d-40a8-a2e4-8048059b43ad-service-ca\") pod \"console-78c8755b99-kbc6p\" (UID: \"0e2abc06-b17d-40a8-a2e4-8048059b43ad\") " pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:35:58.590118 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.590093 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e2abc06-b17d-40a8-a2e4-8048059b43ad-console-config\") pod \"console-78c8755b99-kbc6p\" (UID: \"0e2abc06-b17d-40a8-a2e4-8048059b43ad\") " pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:35:58.591792 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.591763 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e2abc06-b17d-40a8-a2e4-8048059b43ad-console-serving-cert\") pod \"console-78c8755b99-kbc6p\" (UID: \"0e2abc06-b17d-40a8-a2e4-8048059b43ad\") " pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:35:58.591905 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.591861 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e2abc06-b17d-40a8-a2e4-8048059b43ad-console-oauth-config\") pod \"console-78c8755b99-kbc6p\" (UID: \"0e2abc06-b17d-40a8-a2e4-8048059b43ad\") " pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:35:58.600940 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.600916 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvpfd\" (UniqueName: \"kubernetes.io/projected/0e2abc06-b17d-40a8-a2e4-8048059b43ad-kube-api-access-hvpfd\") pod \"console-78c8755b99-kbc6p\" (UID: \"0e2abc06-b17d-40a8-a2e4-8048059b43ad\") " pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:35:58.653550 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.653510 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:35:58.803879 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.803853 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78c8755b99-kbc6p"] Jul 09 13:35:58.804355 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:35:58.804329 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e2abc06_b17d_40a8_a2e4_8048059b43ad.slice/crio-b11a8fa59f872c5a51a3161c3a19922ce1ed9c57c5e3473e9295b7086afd532b WatchSource:0}: Error finding container b11a8fa59f872c5a51a3161c3a19922ce1ed9c57c5e3473e9295b7086afd532b: Status 404 returned error can't find the container with id b11a8fa59f872c5a51a3161c3a19922ce1ed9c57c5e3473e9295b7086afd532b Jul 09 13:35:58.926559 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.926522 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c8755b99-kbc6p" event={"ID":"0e2abc06-b17d-40a8-a2e4-8048059b43ad","Type":"ContainerStarted","Data":"981a6e9bb11f28fe62327082d226951ae353fb2b13476ef097142e8e14e8040a"} Jul 09 13:35:58.926720 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.926606 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c8755b99-kbc6p" event={"ID":"0e2abc06-b17d-40a8-a2e4-8048059b43ad","Type":"ContainerStarted","Data":"b11a8fa59f872c5a51a3161c3a19922ce1ed9c57c5e3473e9295b7086afd532b"} Jul 09 13:35:58.973138 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:35:58.973078 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78c8755b99-kbc6p" podStartSLOduration=0.973061037 podStartE2EDuration="973.061037ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:35:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:35:58.969842537 +0000 UTC m=+376.475144818" watchObservedRunningTime="2026-07-09 13:35:58.973061037 +0000 UTC m=+376.478363304" Jul 09 13:36:08.653959 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:08.653923 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:36:08.654491 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:08.653982 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:36:08.658831 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:08.658804 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:36:08.978728 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:08.978634 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78c8755b99-kbc6p" Jul 09 13:36:09.051499 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:09.051465 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5665b7b4df-6skjd"] Jul 09 13:36:34.075362 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:34.075302 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5665b7b4df-6skjd" podUID="2ed7d06a-8a12-401f-98c1-3e26f38d6c50" containerName="console" containerID="cri-o://a4756037ea0a5c3017bcdd47355d8af7780cdbf07e72f4b6e457f8df2a992597" gracePeriod=15 Jul 09 13:36:34.125828 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:36:34.125784 2573 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ed7d06a_8a12_401f_98c1_3e26f38d6c50.slice/crio-conmon-a4756037ea0a5c3017bcdd47355d8af7780cdbf07e72f4b6e457f8df2a992597.scope\": RecentStats: unable to find data in memory cache]" Jul 09 13:36:34.326968 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:34.326896 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:36:34.438434 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:34.438395 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-console-serving-cert\") pod \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " Jul 09 13:36:34.438634 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:34.438444 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-console-config\") pod \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " Jul 09 13:36:34.438634 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:34.438488 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-trusted-ca-bundle\") pod \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " Jul 09 13:36:34.438634 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:34.438518 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-service-ca\") pod \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " Jul 09 13:36:34.438634 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:34.438540 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4shzd\" (UniqueName: \"kubernetes.io/projected/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-kube-api-access-4shzd\") pod \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " Jul 09 13:36:34.438634 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:34.438590 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-console-oauth-config\") pod \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " Jul 09 13:36:34.438893 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:34.438677 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-oauth-serving-cert\") pod \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\" (UID: \"2ed7d06a-8a12-401f-98c1-3e26f38d6c50\") " Jul 09 13:36:34.438945 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:34.438880 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-service-ca" (OuterVolumeSpecName: "service-ca") pod "2ed7d06a-8a12-401f-98c1-3e26f38d6c50" (UID: "2ed7d06a-8a12-401f-98c1-3e26f38d6c50"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:36:34.438945 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:34.438906 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-console-config" (OuterVolumeSpecName: "console-config") pod "2ed7d06a-8a12-401f-98c1-3e26f38d6c50" (UID: "2ed7d06a-8a12-401f-98c1-3e26f38d6c50"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:36:34.439087 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:34.438942 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2ed7d06a-8a12-401f-98c1-3e26f38d6c50" (UID: "2ed7d06a-8a12-401f-98c1-3e26f38d6c50"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:36:34.439222 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:34.439196 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-trusted-ca-bundle\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:36:34.439222 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:34.439223 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-service-ca\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:36:34.439420 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:34.439237 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-console-config\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:36:34.439420 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:34.439254 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2ed7d06a-8a12-401f-98c1-3e26f38d6c50" (UID: "2ed7d06a-8a12-401f-98c1-3e26f38d6c50"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:36:34.440688 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:34.440664 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2ed7d06a-8a12-401f-98c1-3e26f38d6c50" (UID: "2ed7d06a-8a12-401f-98c1-3e26f38d6c50"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:36:34.440862 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:34.440833 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-kube-api-access-4shzd" (OuterVolumeSpecName: "kube-api-access-4shzd") pod "2ed7d06a-8a12-401f-98c1-3e26f38d6c50" (UID: "2ed7d06a-8a12-401f-98c1-3e26f38d6c50"). InnerVolumeSpecName "kube-api-access-4shzd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 13:36:34.440935 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:34.440901 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2ed7d06a-8a12-401f-98c1-3e26f38d6c50" (UID: "2ed7d06a-8a12-401f-98c1-3e26f38d6c50"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 13:36:34.540645 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:34.540603 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-console-serving-cert\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:36:34.540645 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:34.540635 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4shzd\" (UniqueName: \"kubernetes.io/projected/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-kube-api-access-4shzd\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:36:34.540645 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:34.540645 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-console-oauth-config\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:36:34.540645 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:34.540654 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ed7d06a-8a12-401f-98c1-3e26f38d6c50-oauth-serving-cert\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:36:35.109660 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:35.109621 2573 generic.go:356] "Generic (PLEG): container finished" podID="2ed7d06a-8a12-401f-98c1-3e26f38d6c50" containerID="a4756037ea0a5c3017bcdd47355d8af7780cdbf07e72f4b6e457f8df2a992597" exitCode=0 Jul 09 13:36:35.110070 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:35.109672 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5665b7b4df-6skjd" Jul 09 13:36:35.110070 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:35.109705 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5665b7b4df-6skjd" event={"ID":"2ed7d06a-8a12-401f-98c1-3e26f38d6c50","Type":"ContainerDied","Data":"a4756037ea0a5c3017bcdd47355d8af7780cdbf07e72f4b6e457f8df2a992597"} Jul 09 13:36:35.110070 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:35.109765 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5665b7b4df-6skjd" event={"ID":"2ed7d06a-8a12-401f-98c1-3e26f38d6c50","Type":"ContainerDied","Data":"8ba66ee4f5135aec0bed3ff2794f965c41292fb5d5d49634f91fe4d73b8217de"} Jul 09 13:36:35.110070 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:35.109790 2573 scope.go:117] "RemoveContainer" containerID="a4756037ea0a5c3017bcdd47355d8af7780cdbf07e72f4b6e457f8df2a992597" Jul 09 13:36:35.118632 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:35.118610 2573 scope.go:117] "RemoveContainer" containerID="a4756037ea0a5c3017bcdd47355d8af7780cdbf07e72f4b6e457f8df2a992597" Jul 09 13:36:35.118936 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:36:35.118917 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4756037ea0a5c3017bcdd47355d8af7780cdbf07e72f4b6e457f8df2a992597\": container with ID starting with a4756037ea0a5c3017bcdd47355d8af7780cdbf07e72f4b6e457f8df2a992597 not found: ID does not exist" containerID="a4756037ea0a5c3017bcdd47355d8af7780cdbf07e72f4b6e457f8df2a992597" Jul 09 13:36:35.118986 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:35.118947 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4756037ea0a5c3017bcdd47355d8af7780cdbf07e72f4b6e457f8df2a992597"} err="failed to get container status \"a4756037ea0a5c3017bcdd47355d8af7780cdbf07e72f4b6e457f8df2a992597\": rpc error: code = NotFound desc = could not find container \"a4756037ea0a5c3017bcdd47355d8af7780cdbf07e72f4b6e457f8df2a992597\": container with ID starting with a4756037ea0a5c3017bcdd47355d8af7780cdbf07e72f4b6e457f8df2a992597 not found: ID does not exist" Jul 09 13:36:35.132200 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:35.132166 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5665b7b4df-6skjd"] Jul 09 13:36:35.141790 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:35.141759 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5665b7b4df-6skjd"] Jul 09 13:36:36.945290 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:36.945247 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ed7d06a-8a12-401f-98c1-3e26f38d6c50" path="/var/lib/kubelet/pods/2ed7d06a-8a12-401f-98c1-3e26f38d6c50/volumes" Jul 09 13:36:44.936102 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:44.936070 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g"] Jul 09 13:36:44.936501 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:44.936428 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ed7d06a-8a12-401f-98c1-3e26f38d6c50" containerName="console" Jul 09 13:36:44.936501 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:44.936439 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ed7d06a-8a12-401f-98c1-3e26f38d6c50" containerName="console" Jul 09 13:36:44.936574 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:44.936514 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ed7d06a-8a12-401f-98c1-3e26f38d6c50" containerName="console" Jul 09 13:36:44.940432 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:44.940397 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:44.943592 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:44.943567 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-qvf2x\"" Jul 09 13:36:44.954578 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:44.954541 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g"] Jul 09 13:36:45.029340 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.029297 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/623826de-9ff7-4fdb-a68d-dde99e7babb2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.029521 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.029350 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/623826de-9ff7-4fdb-a68d-dde99e7babb2-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.029521 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.029369 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/623826de-9ff7-4fdb-a68d-dde99e7babb2-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.029521 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.029456 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/623826de-9ff7-4fdb-a68d-dde99e7babb2-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.029521 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.029508 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/623826de-9ff7-4fdb-a68d-dde99e7babb2-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.029656 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.029577 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/623826de-9ff7-4fdb-a68d-dde99e7babb2-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.029656 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.029614 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmlld\" (UniqueName: \"kubernetes.io/projected/623826de-9ff7-4fdb-a68d-dde99e7babb2-kube-api-access-jmlld\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.029767 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.029670 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/623826de-9ff7-4fdb-a68d-dde99e7babb2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.029767 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.029700 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/623826de-9ff7-4fdb-a68d-dde99e7babb2-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.130801 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.130754 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/623826de-9ff7-4fdb-a68d-dde99e7babb2-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.130801 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.130800 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/623826de-9ff7-4fdb-a68d-dde99e7babb2-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.131096 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.130836 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/623826de-9ff7-4fdb-a68d-dde99e7babb2-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.131096 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.130881 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/623826de-9ff7-4fdb-a68d-dde99e7babb2-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.131096 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.130915 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/623826de-9ff7-4fdb-a68d-dde99e7babb2-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.131096 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.130959 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmlld\" (UniqueName: \"kubernetes.io/projected/623826de-9ff7-4fdb-a68d-dde99e7babb2-kube-api-access-jmlld\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.131096 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.131010 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/623826de-9ff7-4fdb-a68d-dde99e7babb2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.131096 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.131075 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/623826de-9ff7-4fdb-a68d-dde99e7babb2-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.131400 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.131125 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/623826de-9ff7-4fdb-a68d-dde99e7babb2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.131400 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.131254 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/623826de-9ff7-4fdb-a68d-dde99e7babb2-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.131400 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.131337 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/623826de-9ff7-4fdb-a68d-dde99e7babb2-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.131564 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.131443 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/623826de-9ff7-4fdb-a68d-dde99e7babb2-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.131564 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.131462 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/623826de-9ff7-4fdb-a68d-dde99e7babb2-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.131787 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.131766 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/623826de-9ff7-4fdb-a68d-dde99e7babb2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.133315 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.133291 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/623826de-9ff7-4fdb-a68d-dde99e7babb2-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.133557 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.133541 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/623826de-9ff7-4fdb-a68d-dde99e7babb2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.142644 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.142612 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/623826de-9ff7-4fdb-a68d-dde99e7babb2-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.143200 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.143183 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmlld\" (UniqueName: \"kubernetes.io/projected/623826de-9ff7-4fdb-a68d-dde99e7babb2-kube-api-access-jmlld\") pod \"maas-default-gateway-openshift-default-56654dcf7-22g4g\" (UID: \"623826de-9ff7-4fdb-a68d-dde99e7babb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.256082 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.255965 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:45.417011 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.416941 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g"] Jul 09 13:36:45.419559 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:36:45.419531 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod623826de_9ff7_4fdb_a68d_dde99e7babb2.slice/crio-cfb961462e9b1d714a9ed448710d933f5128656642c2ec030200c548fc11d911 WatchSource:0}: Error finding container cfb961462e9b1d714a9ed448710d933f5128656642c2ec030200c548fc11d911: Status 404 returned error can't find the container with id cfb961462e9b1d714a9ed448710d933f5128656642c2ec030200c548fc11d911 Jul 09 13:36:45.421702 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.421666 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:36:45.421800 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.421740 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:36:45.421800 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:45.421772 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:36:46.166272 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:46.166235 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" event={"ID":"623826de-9ff7-4fdb-a68d-dde99e7babb2","Type":"ContainerStarted","Data":"1fedfb622c90d8a3616e7610f65c5f1493ea50bf32fb15fd1a62f168398a9fc2"} Jul 09 13:36:46.166712 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:46.166289 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" event={"ID":"623826de-9ff7-4fdb-a68d-dde99e7babb2","Type":"ContainerStarted","Data":"cfb961462e9b1d714a9ed448710d933f5128656642c2ec030200c548fc11d911"} Jul 09 13:36:46.195822 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:46.195763 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" podStartSLOduration=2.195747072 podStartE2EDuration="2.195747072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:36:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:36:46.194186481 +0000 UTC m=+423.699488748" watchObservedRunningTime="2026-07-09 13:36:46.195747072 +0000 UTC m=+423.701049339" Jul 09 13:36:46.256956 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:46.256925 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:46.261796 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:46.261768 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:47.171837 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:47.171804 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:36:47.172784 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:36:47.172761 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:37:00.252149 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:37:00.252114 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:37:00.255728 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:37:00.255705 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-m56nm" Jul 09 13:37:00.258553 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:37:00.258527 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 13:37:00.258686 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:37:00.258527 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 13:37:00.258686 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:37:00.258527 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gc4qp\"" Jul 09 13:37:00.259588 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:37:00.259570 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 09 13:37:00.265686 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:37:00.265660 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:37:00.310314 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:37:00.310276 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:37:00.385786 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:37:00.385748 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/914a5cad-ac5f-4108-98cb-d53fd699049c-config-file\") pod \"limitador-limitador-54d6d988d5-m56nm\" (UID: \"914a5cad-ac5f-4108-98cb-d53fd699049c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-m56nm" Jul 09 13:37:00.385985 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:37:00.385838 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rj9z8\" (UniqueName: \"kubernetes.io/projected/914a5cad-ac5f-4108-98cb-d53fd699049c-kube-api-access-rj9z8\") pod \"limitador-limitador-54d6d988d5-m56nm\" (UID: \"914a5cad-ac5f-4108-98cb-d53fd699049c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-m56nm" Jul 09 13:37:00.486534 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:37:00.486497 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rj9z8\" (UniqueName: \"kubernetes.io/projected/914a5cad-ac5f-4108-98cb-d53fd699049c-kube-api-access-rj9z8\") pod \"limitador-limitador-54d6d988d5-m56nm\" (UID: \"914a5cad-ac5f-4108-98cb-d53fd699049c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-m56nm" Jul 09 13:37:00.486731 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:37:00.486590 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/914a5cad-ac5f-4108-98cb-d53fd699049c-config-file\") pod \"limitador-limitador-54d6d988d5-m56nm\" (UID: \"914a5cad-ac5f-4108-98cb-d53fd699049c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-m56nm" Jul 09 13:37:00.487265 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:37:00.487244 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/914a5cad-ac5f-4108-98cb-d53fd699049c-config-file\") pod \"limitador-limitador-54d6d988d5-m56nm\" (UID: \"914a5cad-ac5f-4108-98cb-d53fd699049c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-m56nm" Jul 09 13:37:00.497280 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:37:00.497251 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rj9z8\" (UniqueName: \"kubernetes.io/projected/914a5cad-ac5f-4108-98cb-d53fd699049c-kube-api-access-rj9z8\") pod \"limitador-limitador-54d6d988d5-m56nm\" (UID: \"914a5cad-ac5f-4108-98cb-d53fd699049c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-m56nm" Jul 09 13:37:00.567106 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:37:00.566978 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-m56nm" Jul 09 13:37:00.709862 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:37:00.709748 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:37:00.712744 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:37:00.712699 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod914a5cad_ac5f_4108_98cb_d53fd699049c.slice/crio-0e588a01ba7bd23f4138f6916d271f12fe513902add52caff60d82b6fbe037ba WatchSource:0}: Error finding container 0e588a01ba7bd23f4138f6916d271f12fe513902add52caff60d82b6fbe037ba: Status 404 returned error can't find the container with id 0e588a01ba7bd23f4138f6916d271f12fe513902add52caff60d82b6fbe037ba Jul 09 13:37:00.714764 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:37:00.714735 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 13:37:01.248298 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:37:01.248108 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-m56nm" event={"ID":"914a5cad-ac5f-4108-98cb-d53fd699049c","Type":"ContainerStarted","Data":"0e588a01ba7bd23f4138f6916d271f12fe513902add52caff60d82b6fbe037ba"} Jul 09 13:37:04.266234 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:37:04.266197 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-m56nm" event={"ID":"914a5cad-ac5f-4108-98cb-d53fd699049c","Type":"ContainerStarted","Data":"c4434caf86cae2be8e0ed15e988aa2dd7a8c81fad7f6bb8c384313f059032ca6"} Jul 09 13:37:04.266635 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:37:04.266262 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-m56nm" Jul 09 13:37:04.290181 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:37:04.290105 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-m56nm" podStartSLOduration=1.740390793 podStartE2EDuration="4.290088695s" totalImagesPullingTime="2.549697902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:37:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:37:04.28950135 +0000 UTC m=+441.794803638" watchObservedRunningTime="2026-07-09 13:37:04.290088695 +0000 UTC m=+441.795390997" Jul 09 13:37:15.272284 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:37:15.272251 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-m56nm" Jul 09 13:38:14.183647 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:38:14.183613 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:38:21.666055 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:38:21.665569 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" event={"ID":"623826de-9ff7-4fdb-a68d-dde99e7babb2","Type":"ContainerDied","Data":"1fedfb622c90d8a3616e7610f65c5f1493ea50bf32fb15fd1a62f168398a9fc2"} Jul 09 13:38:21.666575 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:38:21.666392 2573 scope.go:117] "RemoveContainer" containerID="1fedfb622c90d8a3616e7610f65c5f1493ea50bf32fb15fd1a62f168398a9fc2" Jul 09 13:38:21.670978 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:38:21.666663 2573 generic.go:356] "Generic (PLEG): container finished" podID="623826de-9ff7-4fdb-a68d-dde99e7babb2" containerID="1fedfb622c90d8a3616e7610f65c5f1493ea50bf32fb15fd1a62f168398a9fc2" exitCode=0 Jul 09 13:38:21.672489 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:38:21.672454 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:38:21.672608 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:38:21.672539 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:38:21.672608 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:38:21.672587 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:38:22.674221 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:38:22.674179 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" event={"ID":"623826de-9ff7-4fdb-a68d-dde99e7babb2","Type":"ContainerStarted","Data":"89295501526d5aacf0d44b779d94838015e0a43bfb478906c583f3e251642f37"} Jul 09 13:38:23.256727 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:38:23.256683 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:38:23.256963 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:38:23.256941 2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-22g4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 09 13:38:23.257010 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:38:23.256988 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" podUID="623826de-9ff7-4fdb-a68d-dde99e7babb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 09 13:38:24.256790 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:38:24.256753 2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-22g4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 09 13:38:24.257212 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:38:24.256813 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" podUID="623826de-9ff7-4fdb-a68d-dde99e7babb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 09 13:38:25.256553 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:38:25.256511 2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-22g4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 09 13:38:25.256738 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:38:25.256573 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" podUID="623826de-9ff7-4fdb-a68d-dde99e7babb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 09 13:38:26.257210 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:38:26.257175 2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-22g4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 09 13:38:26.257602 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:38:26.257232 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" podUID="623826de-9ff7-4fdb-a68d-dde99e7babb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 09 13:38:27.260284 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:38:27.260252 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:38:27.260673 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:38:27.260601 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:38:27.261455 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:38:27.261437 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-22g4g" Jul 09 13:38:41.100158 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:38:41.100121 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:38:55.295503 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:38:55.295459 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:39:02.475632 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:39:02.475594 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:39:09.276970 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:39:09.276930 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:39:16.774838 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:39:16.774801 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:39:26.585592 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:39:26.585555 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:39:42.875749 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:39:42.875714 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmms8_3b512d09-0288-4fea-864e-fdc734afb1bd/ovn-acl-logging/0.log" Jul 09 13:39:42.878070 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:39:42.878048 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmms8_3b512d09-0288-4fea-864e-fdc734afb1bd/ovn-acl-logging/0.log" Jul 09 13:40:34.378942 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:40:34.378904 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:40:44.878990 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:40:44.878949 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:40:55.782035 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:40:55.781977 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:41:04.778700 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:41:04.778659 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:41:16.586517 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:41:16.586480 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:41:40.786226 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:41:40.786174 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:41:55.572080 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:41:55.572047 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:42:32.482917 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:42:32.482877 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:42:42.654120 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:42:42.654070 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:42:52.113584 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:42:52.113540 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:42:59.379122 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:42:59.379083 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:43:10.583955 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:43:10.583865 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:43:20.488040 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:43:20.487981 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:43:31.292975 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:43:31.292934 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:43:41.584831 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:43:41.584797 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:44:03.605191 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:44:03.605156 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:44:13.308842 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:44:13.308804 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:44:39.130701 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:44:39.130596 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:44:42.907033 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:44:42.906999 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmms8_3b512d09-0288-4fea-864e-fdc734afb1bd/ovn-acl-logging/0.log" Jul 09 13:44:42.909393 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:44:42.909372 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmms8_3b512d09-0288-4fea-864e-fdc734afb1bd/ovn-acl-logging/0.log" Jul 09 13:44:49.884093 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:44:49.884028 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:45:00.174365 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:45:00.174323 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726745-wdd57"] Jul 09 13:45:00.178133 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:45:00.178107 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726745-wdd57" Jul 09 13:45:00.182447 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:45:00.182426 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 09 13:45:00.182657 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:45:00.182637 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 09 13:45:00.183208 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:45:00.183190 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-z2qnx\"" Jul 09 13:45:00.248717 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:45:00.248677 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726745-wdd57"] Jul 09 13:45:00.302663 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:45:00.302626 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65sww\" (UniqueName: \"kubernetes.io/projected/52f925a0-aa78-4f12-b7cb-056ef767b19d-kube-api-access-65sww\") pod \"maas-api-key-cleanup-29726745-wdd57\" (UID: \"52f925a0-aa78-4f12-b7cb-056ef767b19d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726745-wdd57" Jul 09 13:45:00.404230 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:45:00.404183 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65sww\" (UniqueName: \"kubernetes.io/projected/52f925a0-aa78-4f12-b7cb-056ef767b19d-kube-api-access-65sww\") pod \"maas-api-key-cleanup-29726745-wdd57\" (UID: \"52f925a0-aa78-4f12-b7cb-056ef767b19d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726745-wdd57" Jul 09 13:45:00.416189 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:45:00.416157 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65sww\" (UniqueName: \"kubernetes.io/projected/52f925a0-aa78-4f12-b7cb-056ef767b19d-kube-api-access-65sww\") pod \"maas-api-key-cleanup-29726745-wdd57\" (UID: \"52f925a0-aa78-4f12-b7cb-056ef767b19d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726745-wdd57" Jul 09 13:45:00.489171 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:45:00.489074 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726745-wdd57" Jul 09 13:45:00.630666 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:45:00.630628 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726745-wdd57"] Jul 09 13:45:00.634123 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:45:00.634089 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52f925a0_aa78_4f12_b7cb_056ef767b19d.slice/crio-5c911f4fd1ec111403b981f9c34322c8eb7dcb4a4252543b8b47bc2a9dc31c77 WatchSource:0}: Error finding container 5c911f4fd1ec111403b981f9c34322c8eb7dcb4a4252543b8b47bc2a9dc31c77: Status 404 returned error can't find the container with id 5c911f4fd1ec111403b981f9c34322c8eb7dcb4a4252543b8b47bc2a9dc31c77 Jul 09 13:45:00.636006 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:45:00.635991 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 13:45:00.686699 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:45:00.686664 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726745-wdd57" event={"ID":"52f925a0-aa78-4f12-b7cb-056ef767b19d","Type":"ContainerStarted","Data":"5c911f4fd1ec111403b981f9c34322c8eb7dcb4a4252543b8b47bc2a9dc31c77"} Jul 09 13:45:03.704930 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:45:03.704895 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726745-wdd57" event={"ID":"52f925a0-aa78-4f12-b7cb-056ef767b19d","Type":"ContainerStarted","Data":"8d045e51fd08a08bf4701357dbae7ce2125a237890f120fb2859fa81825ad531"} Jul 09 13:45:03.727604 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:45:03.727554 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726745-wdd57" podStartSLOduration=1.557317442 podStartE2EDuration="3.727537158s" totalImagesPullingTime="2.170219716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:45:03.725267 +0000 UTC m=+921.230569287" watchObservedRunningTime="2026-07-09 13:45:03.727537158 +0000 UTC m=+921.232839425" Jul 09 13:45:04.888523 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:45:04.888489 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:45:22.208719 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:45:22.208680 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:45:32.884394 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:45:32.884354 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:45:42.309808 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:45:42.309769 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:46:00.013246 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:46:00.013206 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726745-wdd57"] Jul 09 13:46:00.013720 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:46:00.013424 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726745-wdd57" podUID="52f925a0-aa78-4f12-b7cb-056ef767b19d" containerName="cleanup" containerID="cri-o://8d045e51fd08a08bf4701357dbae7ce2125a237890f120fb2859fa81825ad531" gracePeriod=30 Jul 09 13:46:30.136903 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:46:30.136865 2573 generic.go:356] "Generic (PLEG): container finished" podID="52f925a0-aa78-4f12-b7cb-056ef767b19d" containerID="8d045e51fd08a08bf4701357dbae7ce2125a237890f120fb2859fa81825ad531" exitCode=137 Jul 09 13:46:30.137342 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:46:30.136943 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726745-wdd57" event={"ID":"52f925a0-aa78-4f12-b7cb-056ef767b19d","Type":"ContainerDied","Data":"8d045e51fd08a08bf4701357dbae7ce2125a237890f120fb2859fa81825ad531"} Jul 09 13:46:30.197193 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:46:30.197161 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726745-wdd57" Jul 09 13:46:30.298289 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:46:30.298184 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-65sww\" (UniqueName: \"kubernetes.io/projected/52f925a0-aa78-4f12-b7cb-056ef767b19d-kube-api-access-65sww\") pod \"52f925a0-aa78-4f12-b7cb-056ef767b19d\" (UID: \"52f925a0-aa78-4f12-b7cb-056ef767b19d\") " Jul 09 13:46:30.300429 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:46:30.300396 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52f925a0-aa78-4f12-b7cb-056ef767b19d-kube-api-access-65sww" (OuterVolumeSpecName: "kube-api-access-65sww") pod "52f925a0-aa78-4f12-b7cb-056ef767b19d" (UID: "52f925a0-aa78-4f12-b7cb-056ef767b19d"). InnerVolumeSpecName "kube-api-access-65sww". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 13:46:30.399882 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:46:30.399842 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65sww\" (UniqueName: \"kubernetes.io/projected/52f925a0-aa78-4f12-b7cb-056ef767b19d-kube-api-access-65sww\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:46:31.142823 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:46:31.142791 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726745-wdd57" Jul 09 13:46:31.143327 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:46:31.142792 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726745-wdd57" event={"ID":"52f925a0-aa78-4f12-b7cb-056ef767b19d","Type":"ContainerDied","Data":"5c911f4fd1ec111403b981f9c34322c8eb7dcb4a4252543b8b47bc2a9dc31c77"} Jul 09 13:46:31.143327 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:46:31.142921 2573 scope.go:117] "RemoveContainer" containerID="8d045e51fd08a08bf4701357dbae7ce2125a237890f120fb2859fa81825ad531" Jul 09 13:46:31.170594 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:46:31.170560 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726745-wdd57"] Jul 09 13:46:31.176307 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:46:31.176279 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726745-wdd57"] Jul 09 13:46:32.945564 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:46:32.945518 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52f925a0-aa78-4f12-b7cb-056ef767b19d" path="/var/lib/kubelet/pods/52f925a0-aa78-4f12-b7cb-056ef767b19d/volumes" Jul 09 13:47:30.897165 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:47:30.897128 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:47:42.884443 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:47:42.884401 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:47:55.281536 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:47:55.281494 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:48:04.889652 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:48:04.889616 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:48:14.678126 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:48:14.678087 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:48:24.723495 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:48:24.723460 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:48:34.281910 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:48:34.281875 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:48:45.380541 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:48:45.380501 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:48:54.687425 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:48:54.687389 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:49:04.595585 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:49:04.595496 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:49:15.590158 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:49:15.590116 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:49:24.402527 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:49:24.402491 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:49:42.938011 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:49:42.937977 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmms8_3b512d09-0288-4fea-864e-fdc734afb1bd/ovn-acl-logging/0.log" Jul 09 13:49:42.942645 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:49:42.942617 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmms8_3b512d09-0288-4fea-864e-fdc734afb1bd/ovn-acl-logging/0.log" Jul 09 13:49:44.977139 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:49:44.977099 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:49:52.383818 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:49:52.383764 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:50:02.489727 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:50:02.489688 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:50:12.392388 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:50:12.392347 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:50:19.412989 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:50:19.412948 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:50:34.987822 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:50:34.987738 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:50:45.780391 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:50:45.780356 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:50:55.284894 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:50:55.284854 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:51:03.184689 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:51:03.184648 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:51:14.579085 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:51:14.579046 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:51:24.083784 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:51:24.083744 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:51:37.301969 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:51:37.301928 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:51:46.581809 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:51:46.581773 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:51:55.993533 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:51:55.993496 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:52:05.786679 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:52:05.786594 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:52:18.344082 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:52:18.344046 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:52:27.701224 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:52:27.701182 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:52:39.992347 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:52:39.992305 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:52:53.098440 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:52:53.098397 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:53:05.480046 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:05.479974 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:53:18.893775 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:18.893733 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7"] Jul 09 13:53:18.894218 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:18.894145 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52f925a0-aa78-4f12-b7cb-056ef767b19d" containerName="cleanup" Jul 09 13:53:18.894218 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:18.894165 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="52f925a0-aa78-4f12-b7cb-056ef767b19d" containerName="cleanup" Jul 09 13:53:18.894293 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:18.894240 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="52f925a0-aa78-4f12-b7cb-056ef767b19d" containerName="cleanup" Jul 09 13:53:18.897375 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:18.897349 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:18.902846 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:18.902821 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b83f08aa-openshift-default-dockercfg-bhmrt\"" Jul 09 13:53:18.930783 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:18.930744 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7"] Jul 09 13:53:18.992232 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:18.992193 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z26lm\" (UniqueName: \"kubernetes.io/projected/9ef3a6de-33c2-44ee-9098-fb60f0534509-kube-api-access-z26lm\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:18.992435 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:18.992240 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-workload-certs\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:18.992435 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:18.992271 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-workload-socket\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:18.992435 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:18.992346 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ef3a6de-33c2-44ee-9098-fb60f0534509-istio-token\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:18.992435 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:18.992425 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ef3a6de-33c2-44ee-9098-fb60f0534509-istiod-ca-cert\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:18.992644 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:18.992469 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ef3a6de-33c2-44ee-9098-fb60f0534509-istio-podinfo\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:18.992644 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:18.992505 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-istio-envoy\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:18.992644 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:18.992538 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-credential-socket\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:18.992644 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:18.992565 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-istio-data\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:19.093429 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:19.093387 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ef3a6de-33c2-44ee-9098-fb60f0534509-istiod-ca-cert\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:19.093618 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:19.093439 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ef3a6de-33c2-44ee-9098-fb60f0534509-istio-podinfo\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:19.093618 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:19.093462 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-istio-envoy\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:19.093618 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:19.093482 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-credential-socket\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:19.093618 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:19.093502 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-istio-data\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:19.093618 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:19.093582 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z26lm\" (UniqueName: \"kubernetes.io/projected/9ef3a6de-33c2-44ee-9098-fb60f0534509-kube-api-access-z26lm\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:19.093618 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:19.093619 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-workload-certs\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:19.093936 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:19.093645 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-workload-socket\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:19.093936 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:19.093679 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ef3a6de-33c2-44ee-9098-fb60f0534509-istio-token\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:19.094080 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:19.093977 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-credential-socket\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:19.094080 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:19.094048 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-istio-data\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:19.094328 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:19.094312 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-workload-socket\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:19.094376 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:19.094309 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-workload-certs\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:19.094411 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:19.094365 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ef3a6de-33c2-44ee-9098-fb60f0534509-istiod-ca-cert\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:19.095971 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:19.095948 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-istio-envoy\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:19.096105 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:19.096086 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ef3a6de-33c2-44ee-9098-fb60f0534509-istio-podinfo\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:19.108416 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:19.108382 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ef3a6de-33c2-44ee-9098-fb60f0534509-istio-token\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:19.108591 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:19.108460 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z26lm\" (UniqueName: \"kubernetes.io/projected/9ef3a6de-33c2-44ee-9098-fb60f0534509-kube-api-access-z26lm\") pod \"e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:19.208297 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:19.208258 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:19.358075 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:19.358048 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7"] Jul 09 13:53:19.360166 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:53:19.360138 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ef3a6de_33c2_44ee_9098_fb60f0534509.slice/crio-bd955e1979d7d48a5b6b4f359fc241320de3475a8b4ea0dc90a29a4b56d755f3 WatchSource:0}: Error finding container bd955e1979d7d48a5b6b4f359fc241320de3475a8b4ea0dc90a29a4b56d755f3: Status 404 returned error can't find the container with id bd955e1979d7d48a5b6b4f359fc241320de3475a8b4ea0dc90a29a4b56d755f3 Jul 09 13:53:19.362365 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:19.362328 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:53:19.362451 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:19.362398 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:53:19.362451 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:19.362428 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:53:20.195180 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:20.195138 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" event={"ID":"9ef3a6de-33c2-44ee-9098-fb60f0534509","Type":"ContainerStarted","Data":"4119b71b428da7597fb040e5b64499a24e4e4b14aeaecdafebb7566dd38a22fa"} Jul 09 13:53:20.195575 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:20.195208 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" event={"ID":"9ef3a6de-33c2-44ee-9098-fb60f0534509","Type":"ContainerStarted","Data":"bd955e1979d7d48a5b6b4f359fc241320de3475a8b4ea0dc90a29a4b56d755f3"} Jul 09 13:53:20.208769 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:20.208740 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:20.213681 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:20.213655 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:20.226857 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:20.226802 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" podStartSLOduration=2.226784363 podStartE2EDuration="2.226784363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:53:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:53:20.225445289 +0000 UTC m=+1417.730747579" watchObservedRunningTime="2026-07-09 13:53:20.226784363 +0000 UTC m=+1417.732086630" Jul 09 13:53:21.200759 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:21.200725 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:21.201669 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:21.201649 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:28.323734 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:28.323695 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7"] Jul 09 13:53:28.324315 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:28.323931 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" podUID="9ef3a6de-33c2-44ee-9098-fb60f0534509" containerName="istio-proxy" containerID="cri-o://4119b71b428da7597fb040e5b64499a24e4e4b14aeaecdafebb7566dd38a22fa" gracePeriod=30 Jul 09 13:53:28.784636 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:28.784594 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:53:33.577394 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.577357 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:33.743545 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.743509 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-workload-certs\") pod \"9ef3a6de-33c2-44ee-9098-fb60f0534509\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " Jul 09 13:53:33.743545 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.743563 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ef3a6de-33c2-44ee-9098-fb60f0534509-istio-token\") pod \"9ef3a6de-33c2-44ee-9098-fb60f0534509\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " Jul 09 13:53:33.743778 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.743582 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-credential-socket\") pod \"9ef3a6de-33c2-44ee-9098-fb60f0534509\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " Jul 09 13:53:33.743778 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.743650 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-workload-socket\") pod \"9ef3a6de-33c2-44ee-9098-fb60f0534509\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " Jul 09 13:53:33.743778 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.743678 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-istio-data\") pod \"9ef3a6de-33c2-44ee-9098-fb60f0534509\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " Jul 09 13:53:33.743778 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.743713 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ef3a6de-33c2-44ee-9098-fb60f0534509-istio-podinfo\") pod \"9ef3a6de-33c2-44ee-9098-fb60f0534509\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " Jul 09 13:53:33.743778 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.743743 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ef3a6de-33c2-44ee-9098-fb60f0534509-istiod-ca-cert\") pod \"9ef3a6de-33c2-44ee-9098-fb60f0534509\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " Jul 09 13:53:33.744072 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.743779 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z26lm\" (UniqueName: \"kubernetes.io/projected/9ef3a6de-33c2-44ee-9098-fb60f0534509-kube-api-access-z26lm\") pod \"9ef3a6de-33c2-44ee-9098-fb60f0534509\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " Jul 09 13:53:33.744072 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.743824 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-istio-envoy\") pod \"9ef3a6de-33c2-44ee-9098-fb60f0534509\" (UID: \"9ef3a6de-33c2-44ee-9098-fb60f0534509\") " Jul 09 13:53:33.744072 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.743825 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9ef3a6de-33c2-44ee-9098-fb60f0534509" (UID: "9ef3a6de-33c2-44ee-9098-fb60f0534509"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:53:33.744072 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.743976 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9ef3a6de-33c2-44ee-9098-fb60f0534509" (UID: "9ef3a6de-33c2-44ee-9098-fb60f0534509"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:53:33.744072 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.744054 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-istio-data" (OuterVolumeSpecName: "istio-data") pod "9ef3a6de-33c2-44ee-9098-fb60f0534509" (UID: "9ef3a6de-33c2-44ee-9098-fb60f0534509"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:53:33.744328 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.744270 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-workload-certs\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:53:33.744328 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.744290 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-credential-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:53:33.744328 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.744305 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-istio-data\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:53:33.744496 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.744333 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ef3a6de-33c2-44ee-9098-fb60f0534509-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9ef3a6de-33c2-44ee-9098-fb60f0534509" (UID: "9ef3a6de-33c2-44ee-9098-fb60f0534509"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:53:33.744496 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.744372 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9ef3a6de-33c2-44ee-9098-fb60f0534509" (UID: "9ef3a6de-33c2-44ee-9098-fb60f0534509"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:53:33.746085 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.745991 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ef3a6de-33c2-44ee-9098-fb60f0534509-istio-token" (OuterVolumeSpecName: "istio-token") pod "9ef3a6de-33c2-44ee-9098-fb60f0534509" (UID: "9ef3a6de-33c2-44ee-9098-fb60f0534509"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 13:53:33.746085 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.746004 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9ef3a6de-33c2-44ee-9098-fb60f0534509-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9ef3a6de-33c2-44ee-9098-fb60f0534509" (UID: "9ef3a6de-33c2-44ee-9098-fb60f0534509"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 13:53:33.746085 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.746055 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ef3a6de-33c2-44ee-9098-fb60f0534509-kube-api-access-z26lm" (OuterVolumeSpecName: "kube-api-access-z26lm") pod "9ef3a6de-33c2-44ee-9098-fb60f0534509" (UID: "9ef3a6de-33c2-44ee-9098-fb60f0534509"). InnerVolumeSpecName "kube-api-access-z26lm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 13:53:33.746286 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.746173 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9ef3a6de-33c2-44ee-9098-fb60f0534509" (UID: "9ef3a6de-33c2-44ee-9098-fb60f0534509"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:53:33.844978 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.844941 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ef3a6de-33c2-44ee-9098-fb60f0534509-istiod-ca-cert\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:53:33.844978 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.844984 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z26lm\" (UniqueName: \"kubernetes.io/projected/9ef3a6de-33c2-44ee-9098-fb60f0534509-kube-api-access-z26lm\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:53:33.845215 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.844994 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-istio-envoy\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:53:33.845215 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.845005 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ef3a6de-33c2-44ee-9098-fb60f0534509-istio-token\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:53:33.845215 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.845013 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ef3a6de-33c2-44ee-9098-fb60f0534509-workload-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:53:33.845215 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:33.845056 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ef3a6de-33c2-44ee-9098-fb60f0534509-istio-podinfo\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:53:34.266904 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:34.266869 2573 generic.go:356] "Generic (PLEG): container finished" podID="9ef3a6de-33c2-44ee-9098-fb60f0534509" containerID="4119b71b428da7597fb040e5b64499a24e4e4b14aeaecdafebb7566dd38a22fa" exitCode=0 Jul 09 13:53:34.267112 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:34.266913 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" Jul 09 13:53:34.267112 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:34.266931 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" event={"ID":"9ef3a6de-33c2-44ee-9098-fb60f0534509","Type":"ContainerDied","Data":"4119b71b428da7597fb040e5b64499a24e4e4b14aeaecdafebb7566dd38a22fa"} Jul 09 13:53:34.267112 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:34.266976 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7" event={"ID":"9ef3a6de-33c2-44ee-9098-fb60f0534509","Type":"ContainerDied","Data":"bd955e1979d7d48a5b6b4f359fc241320de3475a8b4ea0dc90a29a4b56d755f3"} Jul 09 13:53:34.267112 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:34.266997 2573 scope.go:117] "RemoveContainer" containerID="4119b71b428da7597fb040e5b64499a24e4e4b14aeaecdafebb7566dd38a22fa" Jul 09 13:53:34.276614 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:34.276596 2573 scope.go:117] "RemoveContainer" containerID="4119b71b428da7597fb040e5b64499a24e4e4b14aeaecdafebb7566dd38a22fa" Jul 09 13:53:34.276930 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:53:34.276907 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4119b71b428da7597fb040e5b64499a24e4e4b14aeaecdafebb7566dd38a22fa\": container with ID starting with 4119b71b428da7597fb040e5b64499a24e4e4b14aeaecdafebb7566dd38a22fa not found: ID does not exist" containerID="4119b71b428da7597fb040e5b64499a24e4e4b14aeaecdafebb7566dd38a22fa" Jul 09 13:53:34.277045 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:34.276934 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4119b71b428da7597fb040e5b64499a24e4e4b14aeaecdafebb7566dd38a22fa"} err="failed to get container status \"4119b71b428da7597fb040e5b64499a24e4e4b14aeaecdafebb7566dd38a22fa\": rpc error: code = NotFound desc = could not find container \"4119b71b428da7597fb040e5b64499a24e4e4b14aeaecdafebb7566dd38a22fa\": container with ID starting with 4119b71b428da7597fb040e5b64499a24e4e4b14aeaecdafebb7566dd38a22fa not found: ID does not exist" Jul 09 13:53:34.292444 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:34.292333 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7"] Jul 09 13:53:34.303711 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:34.303687 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b83f08aa-openshift-default-774896d56f-zgqb7"] Jul 09 13:53:34.945946 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:34.945908 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ef3a6de-33c2-44ee-9098-fb60f0534509" path="/var/lib/kubelet/pods/9ef3a6de-33c2-44ee-9098-fb60f0534509/volumes" Jul 09 13:53:35.579703 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.579665 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv"] Jul 09 13:53:35.580351 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.580328 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ef3a6de-33c2-44ee-9098-fb60f0534509" containerName="istio-proxy" Jul 09 13:53:35.580444 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.580355 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ef3a6de-33c2-44ee-9098-fb60f0534509" containerName="istio-proxy" Jul 09 13:53:35.580503 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.580447 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ef3a6de-33c2-44ee-9098-fb60f0534509" containerName="istio-proxy" Jul 09 13:53:35.584367 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.584340 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.587406 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.587381 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-13dc4fa5-openshift-default-dockercfg-s8rsp\"" Jul 09 13:53:35.598817 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.598789 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv"] Jul 09 13:53:35.689970 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.689928 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:53:35.763262 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.763220 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r8fz\" (UniqueName: \"kubernetes.io/projected/18867fda-812d-4a85-8a57-5185e09ebe82-kube-api-access-9r8fz\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.763262 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.763266 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-workload-certs\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.763515 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.763300 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18867fda-812d-4a85-8a57-5185e09ebe82-istiod-ca-cert\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.763515 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.763321 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-istio-envoy\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.763515 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.763349 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18867fda-812d-4a85-8a57-5185e09ebe82-istio-token\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.763515 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.763425 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-credential-socket\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.763515 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.763468 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-istio-data\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.763704 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.763522 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-workload-socket\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.763704 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.763547 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18867fda-812d-4a85-8a57-5185e09ebe82-istio-podinfo\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.864705 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.864613 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18867fda-812d-4a85-8a57-5185e09ebe82-istiod-ca-cert\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.864705 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.864652 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-istio-envoy\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.864705 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.864681 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18867fda-812d-4a85-8a57-5185e09ebe82-istio-token\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.864983 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.864814 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-credential-socket\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.864983 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.864870 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-istio-data\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.864983 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.864974 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-workload-socket\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.865192 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.865013 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18867fda-812d-4a85-8a57-5185e09ebe82-istio-podinfo\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.865192 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.865149 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r8fz\" (UniqueName: \"kubernetes.io/projected/18867fda-812d-4a85-8a57-5185e09ebe82-kube-api-access-9r8fz\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.865192 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.865184 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-workload-certs\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.865668 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.865490 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18867fda-812d-4a85-8a57-5185e09ebe82-istiod-ca-cert\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.865668 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.865603 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-credential-socket\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.865927 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.865702 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-istio-data\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.865927 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.865715 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-workload-certs\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.866070 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.865969 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-workload-socket\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.867256 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.867235 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-istio-envoy\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.867496 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.867478 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18867fda-812d-4a85-8a57-5185e09ebe82-istio-podinfo\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.877609 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.877580 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18867fda-812d-4a85-8a57-5185e09ebe82-istio-token\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.878010 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.877988 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r8fz\" (UniqueName: \"kubernetes.io/projected/18867fda-812d-4a85-8a57-5185e09ebe82-kube-api-access-9r8fz\") pod \"e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:35.896726 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:35.896695 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:36.048789 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:36.048757 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv"] Jul 09 13:53:36.050801 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:53:36.050768 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18867fda_812d_4a85_8a57_5185e09ebe82.slice/crio-911ebd6af6652abc17782b177330e4e7be52f46ac93369ed199b41d27186b423 WatchSource:0}: Error finding container 911ebd6af6652abc17782b177330e4e7be52f46ac93369ed199b41d27186b423: Status 404 returned error can't find the container with id 911ebd6af6652abc17782b177330e4e7be52f46ac93369ed199b41d27186b423 Jul 09 13:53:36.052963 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:36.052932 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:53:36.053047 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:36.053002 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:53:36.053092 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:36.053065 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:53:36.281287 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:36.281252 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" event={"ID":"18867fda-812d-4a85-8a57-5185e09ebe82","Type":"ContainerStarted","Data":"6c9a46a4caadc0fe3c8adbaf98b5186452a3caf3ef5375400f2fce92548d4a85"} Jul 09 13:53:36.281480 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:36.281312 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" event={"ID":"18867fda-812d-4a85-8a57-5185e09ebe82","Type":"ContainerStarted","Data":"911ebd6af6652abc17782b177330e4e7be52f46ac93369ed199b41d27186b423"} Jul 09 13:53:36.305951 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:36.305898 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" podStartSLOduration=1.3058755770000001 podStartE2EDuration="1.305875577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:53:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:53:36.304203587 +0000 UTC m=+1433.809505892" watchObservedRunningTime="2026-07-09 13:53:36.305875577 +0000 UTC m=+1433.811177844" Jul 09 13:53:36.897747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:36.897711 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:37.903143 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:37.903108 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:38.291878 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:38.291792 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:38.292818 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:38.292801 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:45.859056 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:45.858997 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv"] Jul 09 13:53:45.859507 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:45.859313 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" podUID="18867fda-812d-4a85-8a57-5185e09ebe82" containerName="istio-proxy" containerID="cri-o://6c9a46a4caadc0fe3c8adbaf98b5186452a3caf3ef5375400f2fce92548d4a85" gracePeriod=30 Jul 09 13:53:49.025803 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.025770 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:49.089148 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.089051 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-workload-certs\") pod \"18867fda-812d-4a85-8a57-5185e09ebe82\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " Jul 09 13:53:49.089148 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.089110 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18867fda-812d-4a85-8a57-5185e09ebe82-istio-podinfo\") pod \"18867fda-812d-4a85-8a57-5185e09ebe82\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " Jul 09 13:53:49.089392 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.089157 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-workload-socket\") pod \"18867fda-812d-4a85-8a57-5185e09ebe82\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " Jul 09 13:53:49.089392 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.089185 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-istio-envoy\") pod \"18867fda-812d-4a85-8a57-5185e09ebe82\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " Jul 09 13:53:49.089392 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.089209 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9r8fz\" (UniqueName: \"kubernetes.io/projected/18867fda-812d-4a85-8a57-5185e09ebe82-kube-api-access-9r8fz\") pod \"18867fda-812d-4a85-8a57-5185e09ebe82\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " Jul 09 13:53:49.089392 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.089234 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-credential-socket\") pod \"18867fda-812d-4a85-8a57-5185e09ebe82\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " Jul 09 13:53:49.089392 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.089259 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18867fda-812d-4a85-8a57-5185e09ebe82-istio-token\") pod \"18867fda-812d-4a85-8a57-5185e09ebe82\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " Jul 09 13:53:49.089392 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.089301 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18867fda-812d-4a85-8a57-5185e09ebe82-istiod-ca-cert\") pod \"18867fda-812d-4a85-8a57-5185e09ebe82\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " Jul 09 13:53:49.089392 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.089324 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-istio-data\") pod \"18867fda-812d-4a85-8a57-5185e09ebe82\" (UID: \"18867fda-812d-4a85-8a57-5185e09ebe82\") " Jul 09 13:53:49.089747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.089506 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "18867fda-812d-4a85-8a57-5185e09ebe82" (UID: "18867fda-812d-4a85-8a57-5185e09ebe82"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:53:49.089747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.089686 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "18867fda-812d-4a85-8a57-5185e09ebe82" (UID: "18867fda-812d-4a85-8a57-5185e09ebe82"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:53:49.089747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.089706 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-workload-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:53:49.089747 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.089700 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18867fda-812d-4a85-8a57-5185e09ebe82-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "18867fda-812d-4a85-8a57-5185e09ebe82" (UID: "18867fda-812d-4a85-8a57-5185e09ebe82"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:53:49.089967 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.089876 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "18867fda-812d-4a85-8a57-5185e09ebe82" (UID: "18867fda-812d-4a85-8a57-5185e09ebe82"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:53:49.090180 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.090158 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-istio-data" (OuterVolumeSpecName: "istio-data") pod "18867fda-812d-4a85-8a57-5185e09ebe82" (UID: "18867fda-812d-4a85-8a57-5185e09ebe82"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:53:49.091557 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.091526 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18867fda-812d-4a85-8a57-5185e09ebe82-kube-api-access-9r8fz" (OuterVolumeSpecName: "kube-api-access-9r8fz") pod "18867fda-812d-4a85-8a57-5185e09ebe82" (UID: "18867fda-812d-4a85-8a57-5185e09ebe82"). InnerVolumeSpecName "kube-api-access-9r8fz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 13:53:49.091676 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.091620 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18867fda-812d-4a85-8a57-5185e09ebe82-istio-token" (OuterVolumeSpecName: "istio-token") pod "18867fda-812d-4a85-8a57-5185e09ebe82" (UID: "18867fda-812d-4a85-8a57-5185e09ebe82"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 13:53:49.091829 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.091812 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/18867fda-812d-4a85-8a57-5185e09ebe82-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "18867fda-812d-4a85-8a57-5185e09ebe82" (UID: "18867fda-812d-4a85-8a57-5185e09ebe82"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 13:53:49.091829 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.091818 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "18867fda-812d-4a85-8a57-5185e09ebe82" (UID: "18867fda-812d-4a85-8a57-5185e09ebe82"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:53:49.191196 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.191161 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-credential-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:53:49.191196 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.191194 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18867fda-812d-4a85-8a57-5185e09ebe82-istio-token\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:53:49.191434 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.191207 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18867fda-812d-4a85-8a57-5185e09ebe82-istiod-ca-cert\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:53:49.191434 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.191217 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-istio-data\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:53:49.191434 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.191228 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-workload-certs\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:53:49.191434 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.191239 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18867fda-812d-4a85-8a57-5185e09ebe82-istio-podinfo\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:53:49.191434 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.191249 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18867fda-812d-4a85-8a57-5185e09ebe82-istio-envoy\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:53:49.191434 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.191278 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9r8fz\" (UniqueName: \"kubernetes.io/projected/18867fda-812d-4a85-8a57-5185e09ebe82-kube-api-access-9r8fz\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:53:49.348614 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.348519 2573 generic.go:356] "Generic (PLEG): container finished" podID="18867fda-812d-4a85-8a57-5185e09ebe82" containerID="6c9a46a4caadc0fe3c8adbaf98b5186452a3caf3ef5375400f2fce92548d4a85" exitCode=0 Jul 09 13:53:49.348614 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.348569 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" Jul 09 13:53:49.348614 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.348604 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" event={"ID":"18867fda-812d-4a85-8a57-5185e09ebe82","Type":"ContainerDied","Data":"6c9a46a4caadc0fe3c8adbaf98b5186452a3caf3ef5375400f2fce92548d4a85"} Jul 09 13:53:49.348828 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.348657 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv" event={"ID":"18867fda-812d-4a85-8a57-5185e09ebe82","Type":"ContainerDied","Data":"911ebd6af6652abc17782b177330e4e7be52f46ac93369ed199b41d27186b423"} Jul 09 13:53:49.348828 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.348679 2573 scope.go:117] "RemoveContainer" containerID="6c9a46a4caadc0fe3c8adbaf98b5186452a3caf3ef5375400f2fce92548d4a85" Jul 09 13:53:49.358886 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.358862 2573 scope.go:117] "RemoveContainer" containerID="6c9a46a4caadc0fe3c8adbaf98b5186452a3caf3ef5375400f2fce92548d4a85" Jul 09 13:53:49.359244 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:53:49.359223 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c9a46a4caadc0fe3c8adbaf98b5186452a3caf3ef5375400f2fce92548d4a85\": container with ID starting with 6c9a46a4caadc0fe3c8adbaf98b5186452a3caf3ef5375400f2fce92548d4a85 not found: ID does not exist" containerID="6c9a46a4caadc0fe3c8adbaf98b5186452a3caf3ef5375400f2fce92548d4a85" Jul 09 13:53:49.359335 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.359254 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c9a46a4caadc0fe3c8adbaf98b5186452a3caf3ef5375400f2fce92548d4a85"} err="failed to get container status \"6c9a46a4caadc0fe3c8adbaf98b5186452a3caf3ef5375400f2fce92548d4a85\": rpc error: code = NotFound desc = could not find container \"6c9a46a4caadc0fe3c8adbaf98b5186452a3caf3ef5375400f2fce92548d4a85\": container with ID starting with 6c9a46a4caadc0fe3c8adbaf98b5186452a3caf3ef5375400f2fce92548d4a85 not found: ID does not exist" Jul 09 13:53:49.373725 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.373687 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv"] Jul 09 13:53:49.380603 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:49.380572 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-13dc4fa5-openshift-default-866fbfc485-ncmhv"] Jul 09 13:53:50.945989 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:50.945954 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18867fda-812d-4a85-8a57-5185e09ebe82" path="/var/lib/kubelet/pods/18867fda-812d-4a85-8a57-5185e09ebe82/volumes" Jul 09 13:53:53.562298 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.562264 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9"] Jul 09 13:53:53.562660 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.562651 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18867fda-812d-4a85-8a57-5185e09ebe82" containerName="istio-proxy" Jul 09 13:53:53.562707 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.562663 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="18867fda-812d-4a85-8a57-5185e09ebe82" containerName="istio-proxy" Jul 09 13:53:53.562766 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.562746 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="18867fda-812d-4a85-8a57-5185e09ebe82" containerName="istio-proxy" Jul 09 13:53:53.567267 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.567243 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.570674 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.570646 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-b98481aa-openshift-default-dockercfg-5xpkx\"" Jul 09 13:53:53.580930 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.580901 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9"] Jul 09 13:53:53.633185 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.633144 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istiod-ca-cert\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.633364 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.633197 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lrtx\" (UniqueName: \"kubernetes.io/projected/a8a6c13d-c94b-496e-9bba-1b35ee09804f-kube-api-access-8lrtx\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.633364 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.633277 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istio-podinfo\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.633364 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.633296 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istio-data\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.633364 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.633341 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istio-envoy\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.633517 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.633375 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istio-token\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.633517 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.633409 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-workload-socket\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.633517 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.633428 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-credential-socket\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.633517 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.633480 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-workload-certs\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.734847 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.734804 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-workload-certs\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.735063 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.734945 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istiod-ca-cert\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.735063 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.734970 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lrtx\" (UniqueName: \"kubernetes.io/projected/a8a6c13d-c94b-496e-9bba-1b35ee09804f-kube-api-access-8lrtx\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.735063 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.735000 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istio-podinfo\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.735063 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.735045 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istio-data\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.735266 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.735198 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istio-envoy\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.735323 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.735278 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istio-token\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.735323 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.735303 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-workload-certs\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.735427 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.735351 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-workload-socket\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.735427 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.735381 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istio-data\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.735427 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.735388 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-credential-socket\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.735716 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.735693 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-credential-socket\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.735716 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.735708 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-workload-socket\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.735848 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.735755 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istiod-ca-cert\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.737547 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.737522 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istio-envoy\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.737684 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.737667 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istio-podinfo\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.749162 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.749128 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istio-token\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.750816 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.750797 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lrtx\" (UniqueName: \"kubernetes.io/projected/a8a6c13d-c94b-496e-9bba-1b35ee09804f-kube-api-access-8lrtx\") pod \"e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:53.878586 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:53.878478 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:54.028816 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:54.028632 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9"] Jul 09 13:53:54.031707 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:53:54.031678 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8a6c13d_c94b_496e_9bba_1b35ee09804f.slice/crio-0cf666c7d7a0f9a329d418935105ccf75c9d6e20ae0d05f4d0c70eea11dd4dae WatchSource:0}: Error finding container 0cf666c7d7a0f9a329d418935105ccf75c9d6e20ae0d05f4d0c70eea11dd4dae: Status 404 returned error can't find the container with id 0cf666c7d7a0f9a329d418935105ccf75c9d6e20ae0d05f4d0c70eea11dd4dae Jul 09 13:53:54.033983 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:54.033944 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:53:54.034129 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:54.034066 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:53:54.034129 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:54.034118 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:53:54.376498 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:54.376463 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" event={"ID":"a8a6c13d-c94b-496e-9bba-1b35ee09804f","Type":"ContainerStarted","Data":"57fb88732350e4ecd2d8ae6451dbc85cb21d8836d86800257ef4e863473bf8b5"} Jul 09 13:53:54.376678 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:54.376521 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" event={"ID":"a8a6c13d-c94b-496e-9bba-1b35ee09804f","Type":"ContainerStarted","Data":"0cf666c7d7a0f9a329d418935105ccf75c9d6e20ae0d05f4d0c70eea11dd4dae"} Jul 09 13:53:54.878845 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:54.878795 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:55.885531 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:55.885493 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:55.913883 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:55.913827 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" podStartSLOduration=2.913810336 podStartE2EDuration="2.913810336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:53:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:53:54.404855566 +0000 UTC m=+1451.910157877" watchObservedRunningTime="2026-07-09 13:53:55.913810336 +0000 UTC m=+1453.419112600" Jul 09 13:53:56.387645 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:56.387608 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:56.388797 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:56.388774 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:53:57.874832 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:53:57.874795 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:54:02.254285 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:02.254249 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9"] Jul 09 13:54:02.254728 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:02.254436 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" podUID="a8a6c13d-c94b-496e-9bba-1b35ee09804f" containerName="istio-proxy" containerID="cri-o://57fb88732350e4ecd2d8ae6451dbc85cb21d8836d86800257ef4e863473bf8b5" gracePeriod=30 Jul 09 13:54:06.015116 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.015050 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:54:06.173776 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.173738 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-workload-certs\") pod \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " Jul 09 13:54:06.173962 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.173790 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istiod-ca-cert\") pod \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " Jul 09 13:54:06.173962 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.173857 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istio-podinfo\") pod \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " Jul 09 13:54:06.173962 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.173887 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lrtx\" (UniqueName: \"kubernetes.io/projected/a8a6c13d-c94b-496e-9bba-1b35ee09804f-kube-api-access-8lrtx\") pod \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " Jul 09 13:54:06.173962 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.173951 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istio-envoy\") pod \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " Jul 09 13:54:06.174226 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.173984 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istio-data\") pod \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " Jul 09 13:54:06.174226 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.174010 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-workload-socket\") pod \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " Jul 09 13:54:06.174226 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.174074 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-credential-socket\") pod \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " Jul 09 13:54:06.174226 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.174099 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a8a6c13d-c94b-496e-9bba-1b35ee09804f" (UID: "a8a6c13d-c94b-496e-9bba-1b35ee09804f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:54:06.174226 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.174117 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istio-token\") pod \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\" (UID: \"a8a6c13d-c94b-496e-9bba-1b35ee09804f\") " Jul 09 13:54:06.174486 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.174370 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a8a6c13d-c94b-496e-9bba-1b35ee09804f" (UID: "a8a6c13d-c94b-496e-9bba-1b35ee09804f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:54:06.174486 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.174396 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a8a6c13d-c94b-496e-9bba-1b35ee09804f" (UID: "a8a6c13d-c94b-496e-9bba-1b35ee09804f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:54:06.174605 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.174549 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a8a6c13d-c94b-496e-9bba-1b35ee09804f" (UID: "a8a6c13d-c94b-496e-9bba-1b35ee09804f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:54:06.174663 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.174628 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istiod-ca-cert\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:54:06.174663 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.174649 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-credential-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:54:06.174773 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.174664 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-workload-certs\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:54:06.175303 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.175272 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istio-data" (OuterVolumeSpecName: "istio-data") pod "a8a6c13d-c94b-496e-9bba-1b35ee09804f" (UID: "a8a6c13d-c94b-496e-9bba-1b35ee09804f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:54:06.176319 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.176295 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istio-token" (OuterVolumeSpecName: "istio-token") pod "a8a6c13d-c94b-496e-9bba-1b35ee09804f" (UID: "a8a6c13d-c94b-496e-9bba-1b35ee09804f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 13:54:06.176425 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.176359 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a8a6c13d-c94b-496e-9bba-1b35ee09804f" (UID: "a8a6c13d-c94b-496e-9bba-1b35ee09804f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 13:54:06.176425 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.176385 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a8a6c13d-c94b-496e-9bba-1b35ee09804f" (UID: "a8a6c13d-c94b-496e-9bba-1b35ee09804f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:54:06.176509 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.176438 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8a6c13d-c94b-496e-9bba-1b35ee09804f-kube-api-access-8lrtx" (OuterVolumeSpecName: "kube-api-access-8lrtx") pod "a8a6c13d-c94b-496e-9bba-1b35ee09804f" (UID: "a8a6c13d-c94b-496e-9bba-1b35ee09804f"). InnerVolumeSpecName "kube-api-access-8lrtx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 13:54:06.276121 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.276090 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istio-podinfo\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:54:06.276121 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.276119 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lrtx\" (UniqueName: \"kubernetes.io/projected/a8a6c13d-c94b-496e-9bba-1b35ee09804f-kube-api-access-8lrtx\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:54:06.276321 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.276130 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istio-envoy\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:54:06.276321 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.276139 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istio-data\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:54:06.276321 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.276147 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8a6c13d-c94b-496e-9bba-1b35ee09804f-workload-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:54:06.276321 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.276154 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8a6c13d-c94b-496e-9bba-1b35ee09804f-istio-token\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:54:06.445260 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.445167 2573 generic.go:356] "Generic (PLEG): container finished" podID="a8a6c13d-c94b-496e-9bba-1b35ee09804f" containerID="57fb88732350e4ecd2d8ae6451dbc85cb21d8836d86800257ef4e863473bf8b5" exitCode=0 Jul 09 13:54:06.445260 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.445214 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" Jul 09 13:54:06.445502 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.445255 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" event={"ID":"a8a6c13d-c94b-496e-9bba-1b35ee09804f","Type":"ContainerDied","Data":"57fb88732350e4ecd2d8ae6451dbc85cb21d8836d86800257ef4e863473bf8b5"} Jul 09 13:54:06.445502 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.445321 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9" event={"ID":"a8a6c13d-c94b-496e-9bba-1b35ee09804f","Type":"ContainerDied","Data":"0cf666c7d7a0f9a329d418935105ccf75c9d6e20ae0d05f4d0c70eea11dd4dae"} Jul 09 13:54:06.445502 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.445351 2573 scope.go:117] "RemoveContainer" containerID="57fb88732350e4ecd2d8ae6451dbc85cb21d8836d86800257ef4e863473bf8b5" Jul 09 13:54:06.454922 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.454897 2573 scope.go:117] "RemoveContainer" containerID="57fb88732350e4ecd2d8ae6451dbc85cb21d8836d86800257ef4e863473bf8b5" Jul 09 13:54:06.455233 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:54:06.455215 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57fb88732350e4ecd2d8ae6451dbc85cb21d8836d86800257ef4e863473bf8b5\": container with ID starting with 57fb88732350e4ecd2d8ae6451dbc85cb21d8836d86800257ef4e863473bf8b5 not found: ID does not exist" containerID="57fb88732350e4ecd2d8ae6451dbc85cb21d8836d86800257ef4e863473bf8b5" Jul 09 13:54:06.455291 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.455242 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57fb88732350e4ecd2d8ae6451dbc85cb21d8836d86800257ef4e863473bf8b5"} err="failed to get container status \"57fb88732350e4ecd2d8ae6451dbc85cb21d8836d86800257ef4e863473bf8b5\": rpc error: code = NotFound desc = could not find container \"57fb88732350e4ecd2d8ae6451dbc85cb21d8836d86800257ef4e863473bf8b5\": container with ID starting with 57fb88732350e4ecd2d8ae6451dbc85cb21d8836d86800257ef4e863473bf8b5 not found: ID does not exist" Jul 09 13:54:06.468815 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.468781 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9"] Jul 09 13:54:06.473288 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.473261 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-b98481aa-openshift-default-674d7c65dc-2klh9"] Jul 09 13:54:06.945614 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:06.945574 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8a6c13d-c94b-496e-9bba-1b35ee09804f" path="/var/lib/kubelet/pods/a8a6c13d-c94b-496e-9bba-1b35ee09804f/volumes" Jul 09 13:54:09.174229 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:09.174189 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:54:42.968727 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:42.968691 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmms8_3b512d09-0288-4fea-864e-fdc734afb1bd/ovn-acl-logging/0.log" Jul 09 13:54:42.974667 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:54:42.974641 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmms8_3b512d09-0288-4fea-864e-fdc734afb1bd/ovn-acl-logging/0.log" Jul 09 13:56:10.973553 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:56:10.973512 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:56:20.174742 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:56:20.174692 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:57:37.264869 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.264832 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj"] Jul 09 13:57:37.265387 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.265233 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8a6c13d-c94b-496e-9bba-1b35ee09804f" containerName="istio-proxy" Jul 09 13:57:37.265387 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.265246 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8a6c13d-c94b-496e-9bba-1b35ee09804f" containerName="istio-proxy" Jul 09 13:57:37.265387 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.265317 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8a6c13d-c94b-496e-9bba-1b35ee09804f" containerName="istio-proxy" Jul 09 13:57:37.268667 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.268647 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.271845 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.271825 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ca4936b3-openshift-default-dockercfg-89ndt\"" Jul 09 13:57:37.334955 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.334908 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj"] Jul 09 13:57:37.353345 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.353306 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istiod-ca-cert\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.353691 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.353658 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-credential-socket\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.353812 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.353739 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istio-data\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.353881 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.353821 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istio-envoy\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.353946 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.353880 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istio-podinfo\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.353946 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.353924 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j67ft\" (UniqueName: \"kubernetes.io/projected/5efe59ea-7491-4d73-b6d8-e83c29740bf4-kube-api-access-j67ft\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.354079 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.353951 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-workload-certs\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.354079 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.353982 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-workload-socket\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.354079 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.354008 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istio-token\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.454678 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.454635 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istio-envoy\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.454863 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.454695 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istio-podinfo\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.454863 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.454733 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j67ft\" (UniqueName: \"kubernetes.io/projected/5efe59ea-7491-4d73-b6d8-e83c29740bf4-kube-api-access-j67ft\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.454863 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.454761 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-workload-certs\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.454863 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.454790 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-workload-socket\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.454863 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.454818 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istio-token\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.455195 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.454880 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istiod-ca-cert\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.455195 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.454920 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-credential-socket\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.455195 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.455153 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istio-data\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.455346 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.455225 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-workload-certs\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.455346 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.455263 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-workload-socket\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.455483 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.455462 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istio-data\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.455534 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.455488 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-credential-socket\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.455627 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.455608 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istiod-ca-cert\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.457292 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.457268 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istio-podinfo\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.457407 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.457356 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istio-envoy\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.464491 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.464467 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istio-token\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.464941 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.464917 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j67ft\" (UniqueName: \"kubernetes.io/projected/5efe59ea-7491-4d73-b6d8-e83c29740bf4-kube-api-access-j67ft\") pod \"e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.579555 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.579455 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:37.726156 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.726124 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj"] Jul 09 13:57:37.728845 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:57:37.728787 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5efe59ea_7491_4d73_b6d8_e83c29740bf4.slice/crio-d8a4e67515ee7e81e47ddbff33f4d75b22fb8bc340a8ec0bc46721f3af4db23b WatchSource:0}: Error finding container d8a4e67515ee7e81e47ddbff33f4d75b22fb8bc340a8ec0bc46721f3af4db23b: Status 404 returned error can't find the container with id d8a4e67515ee7e81e47ddbff33f4d75b22fb8bc340a8ec0bc46721f3af4db23b Jul 09 13:57:37.731267 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.731223 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:57:37.731356 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.731338 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:57:37.731398 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.731386 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:57:37.899215 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:37.899177 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:57:38.522014 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:38.521977 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" event={"ID":"5efe59ea-7491-4d73-b6d8-e83c29740bf4","Type":"ContainerStarted","Data":"74a25c89d83f0b81c9af401bb5e63500701211d70b9c705daa7d57db572f7eb9"} Jul 09 13:57:38.522438 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:38.522048 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" event={"ID":"5efe59ea-7491-4d73-b6d8-e83c29740bf4","Type":"ContainerStarted","Data":"d8a4e67515ee7e81e47ddbff33f4d75b22fb8bc340a8ec0bc46721f3af4db23b"} Jul 09 13:57:38.552408 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:38.552351 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" podStartSLOduration=1.552331211 podStartE2EDuration="1.552331211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:57:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:57:38.549420832 +0000 UTC m=+1676.054723099" watchObservedRunningTime="2026-07-09 13:57:38.552331211 +0000 UTC m=+1676.057633480" Jul 09 13:57:38.580454 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:38.580413 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:38.585446 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:38.585416 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:39.527504 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:39.527456 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:39.528556 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:39.528535 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:46.516617 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:46.516572 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:57:54.532080 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:54.532049 2573 patch_prober.go:28] interesting pod/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": read tcp 10.134.0.2:53222->10.134.0.39:15021: read: connection reset by peer" start-of-body= Jul 09 13:57:54.532567 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:54.532128 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" podUID="5efe59ea-7491-4d73-b6d8-e83c29740bf4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": read tcp 10.134.0.2:53222->10.134.0.39:15021: read: connection reset by peer" Jul 09 13:57:54.609717 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:54.609684 2573 generic.go:356] "Generic (PLEG): container finished" podID="5efe59ea-7491-4d73-b6d8-e83c29740bf4" containerID="74a25c89d83f0b81c9af401bb5e63500701211d70b9c705daa7d57db572f7eb9" exitCode=0 Jul 09 13:57:54.609892 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:54.609759 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" event={"ID":"5efe59ea-7491-4d73-b6d8-e83c29740bf4","Type":"ContainerDied","Data":"74a25c89d83f0b81c9af401bb5e63500701211d70b9c705daa7d57db572f7eb9"} Jul 09 13:57:54.610203 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:54.610188 2573 scope.go:117] "RemoveContainer" containerID="74a25c89d83f0b81c9af401bb5e63500701211d70b9c705daa7d57db572f7eb9" Jul 09 13:57:54.611275 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:54.611241 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:57:54.611394 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:54.611315 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:57:54.611394 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:54.611347 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:57:55.620648 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:55.620602 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" event={"ID":"5efe59ea-7491-4d73-b6d8-e83c29740bf4","Type":"ContainerStarted","Data":"56b7dac457424e6f727165c4108e748871cb6dcbdbe59d0ae3c70d5e4b52aab7"} Jul 09 13:57:56.579982 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:56.579934 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:57:56.580247 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:56.580221 2573 patch_prober.go:28] interesting pod/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 09 13:57:56.580671 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:56.580634 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" podUID="5efe59ea-7491-4d73-b6d8-e83c29740bf4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 09 13:57:57.580899 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:57.580856 2573 patch_prober.go:28] interesting pod/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 09 13:57:57.581446 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:57.580937 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" podUID="5efe59ea-7491-4d73-b6d8-e83c29740bf4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 09 13:57:58.066904 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:58.066863 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:57:58.579969 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:58.579929 2573 patch_prober.go:28] interesting pod/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 09 13:57:58.580191 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:58.579989 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" podUID="5efe59ea-7491-4d73-b6d8-e83c29740bf4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 09 13:57:59.579951 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:59.579910 2573 patch_prober.go:28] interesting pod/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 09 13:57:59.580375 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:57:59.579969 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" podUID="5efe59ea-7491-4d73-b6d8-e83c29740bf4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 09 13:58:00.417817 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:00.417780 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj"] Jul 09 13:58:00.418074 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:00.417969 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" podUID="5efe59ea-7491-4d73-b6d8-e83c29740bf4" containerName="istio-proxy" containerID="cri-o://56b7dac457424e6f727165c4108e748871cb6dcbdbe59d0ae3c70d5e4b52aab7" gracePeriod=30 Jul 09 13:58:05.676114 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.676080 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:58:05.683740 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.683651 2573 generic.go:356] "Generic (PLEG): container finished" podID="5efe59ea-7491-4d73-b6d8-e83c29740bf4" containerID="56b7dac457424e6f727165c4108e748871cb6dcbdbe59d0ae3c70d5e4b52aab7" exitCode=0 Jul 09 13:58:05.683740 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.683702 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" Jul 09 13:58:05.683922 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.683736 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" event={"ID":"5efe59ea-7491-4d73-b6d8-e83c29740bf4","Type":"ContainerDied","Data":"56b7dac457424e6f727165c4108e748871cb6dcbdbe59d0ae3c70d5e4b52aab7"} Jul 09 13:58:05.683922 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.683793 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj" event={"ID":"5efe59ea-7491-4d73-b6d8-e83c29740bf4","Type":"ContainerDied","Data":"d8a4e67515ee7e81e47ddbff33f4d75b22fb8bc340a8ec0bc46721f3af4db23b"} Jul 09 13:58:05.683922 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.683816 2573 scope.go:117] "RemoveContainer" containerID="56b7dac457424e6f727165c4108e748871cb6dcbdbe59d0ae3c70d5e4b52aab7" Jul 09 13:58:05.692607 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.692588 2573 scope.go:117] "RemoveContainer" containerID="74a25c89d83f0b81c9af401bb5e63500701211d70b9c705daa7d57db572f7eb9" Jul 09 13:58:05.703238 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.703212 2573 scope.go:117] "RemoveContainer" containerID="56b7dac457424e6f727165c4108e748871cb6dcbdbe59d0ae3c70d5e4b52aab7" Jul 09 13:58:05.703595 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:58:05.703571 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56b7dac457424e6f727165c4108e748871cb6dcbdbe59d0ae3c70d5e4b52aab7\": container with ID starting with 56b7dac457424e6f727165c4108e748871cb6dcbdbe59d0ae3c70d5e4b52aab7 not found: ID does not exist" containerID="56b7dac457424e6f727165c4108e748871cb6dcbdbe59d0ae3c70d5e4b52aab7" Jul 09 13:58:05.703698 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.703604 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56b7dac457424e6f727165c4108e748871cb6dcbdbe59d0ae3c70d5e4b52aab7"} err="failed to get container status \"56b7dac457424e6f727165c4108e748871cb6dcbdbe59d0ae3c70d5e4b52aab7\": rpc error: code = NotFound desc = could not find container \"56b7dac457424e6f727165c4108e748871cb6dcbdbe59d0ae3c70d5e4b52aab7\": container with ID starting with 56b7dac457424e6f727165c4108e748871cb6dcbdbe59d0ae3c70d5e4b52aab7 not found: ID does not exist" Jul 09 13:58:05.703698 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.703623 2573 scope.go:117] "RemoveContainer" containerID="74a25c89d83f0b81c9af401bb5e63500701211d70b9c705daa7d57db572f7eb9" Jul 09 13:58:05.703861 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:58:05.703845 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74a25c89d83f0b81c9af401bb5e63500701211d70b9c705daa7d57db572f7eb9\": container with ID starting with 74a25c89d83f0b81c9af401bb5e63500701211d70b9c705daa7d57db572f7eb9 not found: ID does not exist" containerID="74a25c89d83f0b81c9af401bb5e63500701211d70b9c705daa7d57db572f7eb9" Jul 09 13:58:05.703902 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.703883 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74a25c89d83f0b81c9af401bb5e63500701211d70b9c705daa7d57db572f7eb9"} err="failed to get container status \"74a25c89d83f0b81c9af401bb5e63500701211d70b9c705daa7d57db572f7eb9\": rpc error: code = NotFound desc = could not find container \"74a25c89d83f0b81c9af401bb5e63500701211d70b9c705daa7d57db572f7eb9\": container with ID starting with 74a25c89d83f0b81c9af401bb5e63500701211d70b9c705daa7d57db572f7eb9 not found: ID does not exist" Jul 09 13:58:05.737335 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.737297 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istio-data\") pod \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " Jul 09 13:58:05.737484 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.737348 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j67ft\" (UniqueName: \"kubernetes.io/projected/5efe59ea-7491-4d73-b6d8-e83c29740bf4-kube-api-access-j67ft\") pod \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " Jul 09 13:58:05.737484 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.737372 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istio-envoy\") pod \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " Jul 09 13:58:05.737484 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.737451 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-credential-socket\") pod \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " Jul 09 13:58:05.737484 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.737474 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istio-podinfo\") pod \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " Jul 09 13:58:05.737699 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.737497 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istio-token\") pod \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " Jul 09 13:58:05.737699 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.737525 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-workload-socket\") pod \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " Jul 09 13:58:05.737830 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.737792 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5efe59ea-7491-4d73-b6d8-e83c29740bf4" (UID: "5efe59ea-7491-4d73-b6d8-e83c29740bf4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:58:05.737830 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.737809 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5efe59ea-7491-4d73-b6d8-e83c29740bf4" (UID: "5efe59ea-7491-4d73-b6d8-e83c29740bf4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:58:05.737958 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.737859 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istiod-ca-cert\") pod \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " Jul 09 13:58:05.737958 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.737919 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-workload-certs\") pod \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\" (UID: \"5efe59ea-7491-4d73-b6d8-e83c29740bf4\") " Jul 09 13:58:05.738181 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.738142 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5efe59ea-7491-4d73-b6d8-e83c29740bf4" (UID: "5efe59ea-7491-4d73-b6d8-e83c29740bf4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:58:05.738334 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.738180 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5efe59ea-7491-4d73-b6d8-e83c29740bf4" (UID: "5efe59ea-7491-4d73-b6d8-e83c29740bf4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:58:05.738408 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.738343 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istio-data" (OuterVolumeSpecName: "istio-data") pod "5efe59ea-7491-4d73-b6d8-e83c29740bf4" (UID: "5efe59ea-7491-4d73-b6d8-e83c29740bf4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:58:05.738408 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.738366 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-credential-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:58:05.738408 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.738387 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-workload-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:58:05.738588 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.738407 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istiod-ca-cert\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:58:05.738588 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.738422 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-workload-certs\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:58:05.739765 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.739737 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5efe59ea-7491-4d73-b6d8-e83c29740bf4-kube-api-access-j67ft" (OuterVolumeSpecName: "kube-api-access-j67ft") pod "5efe59ea-7491-4d73-b6d8-e83c29740bf4" (UID: "5efe59ea-7491-4d73-b6d8-e83c29740bf4"). InnerVolumeSpecName "kube-api-access-j67ft". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 13:58:05.740081 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.740055 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istio-token" (OuterVolumeSpecName: "istio-token") pod "5efe59ea-7491-4d73-b6d8-e83c29740bf4" (UID: "5efe59ea-7491-4d73-b6d8-e83c29740bf4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 13:58:05.740175 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.740138 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5efe59ea-7491-4d73-b6d8-e83c29740bf4" (UID: "5efe59ea-7491-4d73-b6d8-e83c29740bf4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:58:05.740175 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.740157 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5efe59ea-7491-4d73-b6d8-e83c29740bf4" (UID: "5efe59ea-7491-4d73-b6d8-e83c29740bf4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 13:58:05.839754 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.839714 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istio-token\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:58:05.839754 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.839745 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istio-data\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:58:05.839754 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.839754 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j67ft\" (UniqueName: \"kubernetes.io/projected/5efe59ea-7491-4d73-b6d8-e83c29740bf4-kube-api-access-j67ft\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:58:05.839754 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.839763 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istio-envoy\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:58:05.840081 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:05.839772 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5efe59ea-7491-4d73-b6d8-e83c29740bf4-istio-podinfo\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:58:06.008346 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:06.008308 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj"] Jul 09 13:58:06.013449 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:06.013419 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ca4936b3-openshift-default-686f88f75-lmlnj"] Jul 09 13:58:06.945397 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:06.945364 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5efe59ea-7491-4d73-b6d8-e83c29740bf4" path="/var/lib/kubelet/pods/5efe59ea-7491-4d73-b6d8-e83c29740bf4/volumes" Jul 09 13:58:21.576640 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.576598 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg"] Jul 09 13:58:21.577090 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.576974 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5efe59ea-7491-4d73-b6d8-e83c29740bf4" containerName="istio-proxy" Jul 09 13:58:21.577090 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.576984 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5efe59ea-7491-4d73-b6d8-e83c29740bf4" containerName="istio-proxy" Jul 09 13:58:21.577090 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.576992 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5efe59ea-7491-4d73-b6d8-e83c29740bf4" containerName="istio-proxy" Jul 09 13:58:21.577090 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.576997 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5efe59ea-7491-4d73-b6d8-e83c29740bf4" containerName="istio-proxy" Jul 09 13:58:21.577233 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.577105 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="5efe59ea-7491-4d73-b6d8-e83c29740bf4" containerName="istio-proxy" Jul 09 13:58:21.577233 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.577116 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="5efe59ea-7491-4d73-b6d8-e83c29740bf4" containerName="istio-proxy" Jul 09 13:58:21.578994 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.578976 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.581772 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.581749 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-d7d5695b-openshift-default-dockercfg-slqwt\"" Jul 09 13:58:21.595004 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.594972 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg"] Jul 09 13:58:21.698984 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.698937 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-credential-socket\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.698984 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.698989 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3af93efd-b94f-4d10-a648-c4ae04d61115-istio-podinfo\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.699253 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.699012 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkrcm\" (UniqueName: \"kubernetes.io/projected/3af93efd-b94f-4d10-a648-c4ae04d61115-kube-api-access-gkrcm\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.699253 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.699099 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-istio-data\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.699253 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.699136 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-workload-socket\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.699253 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.699160 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3af93efd-b94f-4d10-a648-c4ae04d61115-istiod-ca-cert\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.699253 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.699202 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-istio-envoy\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.699253 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.699248 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3af93efd-b94f-4d10-a648-c4ae04d61115-istio-token\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.699457 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.699307 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-workload-certs\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.800173 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.800135 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-istio-envoy\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.800377 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.800195 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3af93efd-b94f-4d10-a648-c4ae04d61115-istio-token\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.800377 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.800220 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-workload-certs\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.800377 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.800324 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-credential-socket\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.800621 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.800388 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3af93efd-b94f-4d10-a648-c4ae04d61115-istio-podinfo\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.800621 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.800416 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkrcm\" (UniqueName: \"kubernetes.io/projected/3af93efd-b94f-4d10-a648-c4ae04d61115-kube-api-access-gkrcm\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.800621 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.800454 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-istio-data\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.800790 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.800584 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-workload-socket\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.800790 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.800706 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3af93efd-b94f-4d10-a648-c4ae04d61115-istiod-ca-cert\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.800894 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.800788 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-istio-data\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.800894 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.800725 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-credential-socket\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.800894 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.800603 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-workload-certs\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.800894 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.800879 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-workload-socket\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.801384 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.801348 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3af93efd-b94f-4d10-a648-c4ae04d61115-istiod-ca-cert\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.802609 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.802589 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-istio-envoy\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.802904 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.802882 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3af93efd-b94f-4d10-a648-c4ae04d61115-istio-podinfo\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.813225 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.813190 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkrcm\" (UniqueName: \"kubernetes.io/projected/3af93efd-b94f-4d10-a648-c4ae04d61115-kube-api-access-gkrcm\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.813550 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.813530 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3af93efd-b94f-4d10-a648-c4ae04d61115-istio-token\") pod \"e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:21.890547 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:21.890513 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:22.039310 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:22.039279 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg"] Jul 09 13:58:22.041376 ip-10-0-142-87 kubenswrapper[2573]: W0709 13:58:22.041351 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3af93efd_b94f_4d10_a648_c4ae04d61115.slice/crio-9f7fe07f8b642b6f8b3486bc1e1158f8d475f43df8d530f28cfe6bff80c5c01d WatchSource:0}: Error finding container 9f7fe07f8b642b6f8b3486bc1e1158f8d475f43df8d530f28cfe6bff80c5c01d: Status 404 returned error can't find the container with id 9f7fe07f8b642b6f8b3486bc1e1158f8d475f43df8d530f28cfe6bff80c5c01d Jul 09 13:58:22.043421 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:22.043391 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:58:22.043493 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:22.043464 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:58:22.043527 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:22.043492 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:58:22.773271 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:22.773238 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" event={"ID":"3af93efd-b94f-4d10-a648-c4ae04d61115","Type":"ContainerStarted","Data":"4426c674ed95e65e8480b4dca698df9f46c2115a1b3a1eb836df8ae4546b66f1"} Jul 09 13:58:22.773828 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:22.773289 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" event={"ID":"3af93efd-b94f-4d10-a648-c4ae04d61115","Type":"ContainerStarted","Data":"9f7fe07f8b642b6f8b3486bc1e1158f8d475f43df8d530f28cfe6bff80c5c01d"} Jul 09 13:58:22.800174 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:22.800114 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" podStartSLOduration=1.8000951490000001 podStartE2EDuration="1.800095149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:58:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:58:22.798819111 +0000 UTC m=+1720.304121380" watchObservedRunningTime="2026-07-09 13:58:22.800095149 +0000 UTC m=+1720.305397411" Jul 09 13:58:22.890666 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:22.890624 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:23.896260 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:23.896216 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:24.784681 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:24.784640 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:24.785827 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:24.785799 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:29.581731 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:29.581683 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:58:37.850895 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:37.850799 2573 generic.go:356] "Generic (PLEG): container finished" podID="3af93efd-b94f-4d10-a648-c4ae04d61115" containerID="4426c674ed95e65e8480b4dca698df9f46c2115a1b3a1eb836df8ae4546b66f1" exitCode=0 Jul 09 13:58:37.850895 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:37.850872 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" event={"ID":"3af93efd-b94f-4d10-a648-c4ae04d61115","Type":"ContainerDied","Data":"4426c674ed95e65e8480b4dca698df9f46c2115a1b3a1eb836df8ae4546b66f1"} Jul 09 13:58:37.851382 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:37.851298 2573 scope.go:117] "RemoveContainer" containerID="4426c674ed95e65e8480b4dca698df9f46c2115a1b3a1eb836df8ae4546b66f1" Jul 09 13:58:37.852317 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:37.852289 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:58:37.852388 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:37.852358 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:58:37.852427 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:37.852389 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 13:58:38.475437 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:38.475394 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:58:38.859266 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:38.859175 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" event={"ID":"3af93efd-b94f-4d10-a648-c4ae04d61115","Type":"ContainerStarted","Data":"584637deba30a56c2a6dfb8ff8a2cae1d2193906f542d09c71abb3c020118bba"} Jul 09 13:58:38.891425 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:38.891393 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:38.892238 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:38.892213 2573 patch_prober.go:28] interesting pod/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 09 13:58:38.892442 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:38.892417 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" podUID="3af93efd-b94f-4d10-a648-c4ae04d61115" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 09 13:58:39.785378 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:39.785286 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:39.891835 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:39.891799 2573 patch_prober.go:28] interesting pod/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 09 13:58:39.892872 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:39.892447 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" podUID="3af93efd-b94f-4d10-a648-c4ae04d61115" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 09 13:58:40.891367 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:40.891325 2573 patch_prober.go:28] interesting pod/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 09 13:58:40.891592 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:40.891398 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" podUID="3af93efd-b94f-4d10-a648-c4ae04d61115" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 09 13:58:41.890994 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:41.890943 2573 patch_prober.go:28] interesting pod/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 09 13:58:41.891533 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:41.891033 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" podUID="3af93efd-b94f-4d10-a648-c4ae04d61115" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 09 13:58:42.950266 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:42.950217 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:42.951547 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:42.951526 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:58:50.483696 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:58:50.483654 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:59:01.976249 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:01.976206 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:59:12.276008 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:12.275969 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:59:43.002881 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:43.002845 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmms8_3b512d09-0288-4fea-864e-fdc734afb1bd/ovn-acl-logging/0.log" Jul 09 13:59:43.009701 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:43.009677 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmms8_3b512d09-0288-4fea-864e-fdc734afb1bd/ovn-acl-logging/0.log" Jul 09 13:59:51.071589 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:51.071495 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 13:59:53.491275 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:53.491233 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg"] Jul 09 13:59:53.491654 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:53.491486 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" podUID="3af93efd-b94f-4d10-a648-c4ae04d61115" containerName="istio-proxy" containerID="cri-o://584637deba30a56c2a6dfb8ff8a2cae1d2193906f542d09c71abb3c020118bba" gracePeriod=30 Jul 09 13:59:57.953491 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:57.953440 2573 patch_prober.go:28] interesting pod/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 13:59:57.953895 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:57.953511 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" podUID="3af93efd-b94f-4d10-a648-c4ae04d61115" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 13:59:58.753786 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.753714 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:59:58.868289 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.868254 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-istio-envoy\") pod \"3af93efd-b94f-4d10-a648-c4ae04d61115\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " Jul 09 13:59:58.868289 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.868299 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-workload-certs\") pod \"3af93efd-b94f-4d10-a648-c4ae04d61115\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " Jul 09 13:59:58.868540 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.868333 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-credential-socket\") pod \"3af93efd-b94f-4d10-a648-c4ae04d61115\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " Jul 09 13:59:58.868540 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.868355 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3af93efd-b94f-4d10-a648-c4ae04d61115-istio-podinfo\") pod \"3af93efd-b94f-4d10-a648-c4ae04d61115\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " Jul 09 13:59:58.868540 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.868394 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-workload-socket\") pod \"3af93efd-b94f-4d10-a648-c4ae04d61115\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " Jul 09 13:59:58.868540 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.868413 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3af93efd-b94f-4d10-a648-c4ae04d61115-istiod-ca-cert\") pod \"3af93efd-b94f-4d10-a648-c4ae04d61115\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " Jul 09 13:59:58.868540 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.868457 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3af93efd-b94f-4d10-a648-c4ae04d61115-istio-token\") pod \"3af93efd-b94f-4d10-a648-c4ae04d61115\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " Jul 09 13:59:58.868540 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.868483 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-istio-data\") pod \"3af93efd-b94f-4d10-a648-c4ae04d61115\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " Jul 09 13:59:58.868540 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.868527 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkrcm\" (UniqueName: \"kubernetes.io/projected/3af93efd-b94f-4d10-a648-c4ae04d61115-kube-api-access-gkrcm\") pod \"3af93efd-b94f-4d10-a648-c4ae04d61115\" (UID: \"3af93efd-b94f-4d10-a648-c4ae04d61115\") " Jul 09 13:59:58.869304 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.868590 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3af93efd-b94f-4d10-a648-c4ae04d61115" (UID: "3af93efd-b94f-4d10-a648-c4ae04d61115"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:59:58.869304 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.868646 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3af93efd-b94f-4d10-a648-c4ae04d61115" (UID: "3af93efd-b94f-4d10-a648-c4ae04d61115"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:59:58.869304 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.868706 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3af93efd-b94f-4d10-a648-c4ae04d61115" (UID: "3af93efd-b94f-4d10-a648-c4ae04d61115"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:59:58.869304 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.869048 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3af93efd-b94f-4d10-a648-c4ae04d61115-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3af93efd-b94f-4d10-a648-c4ae04d61115" (UID: "3af93efd-b94f-4d10-a648-c4ae04d61115"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 13:59:58.869304 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.869232 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-workload-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:59:58.869304 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.869254 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3af93efd-b94f-4d10-a648-c4ae04d61115-istiod-ca-cert\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:59:58.869304 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.869269 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-workload-certs\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:59:58.869304 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.869282 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-credential-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:59:58.869582 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.869328 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-istio-data" (OuterVolumeSpecName: "istio-data") pod "3af93efd-b94f-4d10-a648-c4ae04d61115" (UID: "3af93efd-b94f-4d10-a648-c4ae04d61115"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:59:58.870674 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.870651 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3af93efd-b94f-4d10-a648-c4ae04d61115" (UID: "3af93efd-b94f-4d10-a648-c4ae04d61115"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 13:59:58.870906 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.870889 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3af93efd-b94f-4d10-a648-c4ae04d61115-istio-token" (OuterVolumeSpecName: "istio-token") pod "3af93efd-b94f-4d10-a648-c4ae04d61115" (UID: "3af93efd-b94f-4d10-a648-c4ae04d61115"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 13:59:58.871194 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.871170 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3af93efd-b94f-4d10-a648-c4ae04d61115-kube-api-access-gkrcm" (OuterVolumeSpecName: "kube-api-access-gkrcm") pod "3af93efd-b94f-4d10-a648-c4ae04d61115" (UID: "3af93efd-b94f-4d10-a648-c4ae04d61115"). InnerVolumeSpecName "kube-api-access-gkrcm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 13:59:58.871299 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.871275 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3af93efd-b94f-4d10-a648-c4ae04d61115-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3af93efd-b94f-4d10-a648-c4ae04d61115" (UID: "3af93efd-b94f-4d10-a648-c4ae04d61115"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 13:59:58.970490 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.970451 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3af93efd-b94f-4d10-a648-c4ae04d61115-istio-podinfo\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:59:58.970490 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.970486 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3af93efd-b94f-4d10-a648-c4ae04d61115-istio-token\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:59:58.970490 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.970494 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-istio-data\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:59:58.970918 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.970502 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkrcm\" (UniqueName: \"kubernetes.io/projected/3af93efd-b94f-4d10-a648-c4ae04d61115-kube-api-access-gkrcm\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:59:58.970918 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:58.970512 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3af93efd-b94f-4d10-a648-c4ae04d61115-istio-envoy\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 13:59:59.278986 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:59.278949 2573 generic.go:356] "Generic (PLEG): container finished" podID="3af93efd-b94f-4d10-a648-c4ae04d61115" containerID="584637deba30a56c2a6dfb8ff8a2cae1d2193906f542d09c71abb3c020118bba" exitCode=0 Jul 09 13:59:59.279191 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:59.279004 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" Jul 09 13:59:59.279191 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:59.279060 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" event={"ID":"3af93efd-b94f-4d10-a648-c4ae04d61115","Type":"ContainerDied","Data":"584637deba30a56c2a6dfb8ff8a2cae1d2193906f542d09c71abb3c020118bba"} Jul 09 13:59:59.279191 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:59.279119 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg" event={"ID":"3af93efd-b94f-4d10-a648-c4ae04d61115","Type":"ContainerDied","Data":"9f7fe07f8b642b6f8b3486bc1e1158f8d475f43df8d530f28cfe6bff80c5c01d"} Jul 09 13:59:59.279191 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:59.279150 2573 scope.go:117] "RemoveContainer" containerID="584637deba30a56c2a6dfb8ff8a2cae1d2193906f542d09c71abb3c020118bba" Jul 09 13:59:59.290321 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:59.290297 2573 scope.go:117] "RemoveContainer" containerID="4426c674ed95e65e8480b4dca698df9f46c2115a1b3a1eb836df8ae4546b66f1" Jul 09 13:59:59.299474 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:59.299447 2573 scope.go:117] "RemoveContainer" containerID="584637deba30a56c2a6dfb8ff8a2cae1d2193906f542d09c71abb3c020118bba" Jul 09 13:59:59.299595 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:59.299557 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg"] Jul 09 13:59:59.300357 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:59:59.300319 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"584637deba30a56c2a6dfb8ff8a2cae1d2193906f542d09c71abb3c020118bba\": container with ID starting with 584637deba30a56c2a6dfb8ff8a2cae1d2193906f542d09c71abb3c020118bba not found: ID does not exist" containerID="584637deba30a56c2a6dfb8ff8a2cae1d2193906f542d09c71abb3c020118bba" Jul 09 13:59:59.300472 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:59.300361 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"584637deba30a56c2a6dfb8ff8a2cae1d2193906f542d09c71abb3c020118bba"} err="failed to get container status \"584637deba30a56c2a6dfb8ff8a2cae1d2193906f542d09c71abb3c020118bba\": rpc error: code = NotFound desc = could not find container \"584637deba30a56c2a6dfb8ff8a2cae1d2193906f542d09c71abb3c020118bba\": container with ID starting with 584637deba30a56c2a6dfb8ff8a2cae1d2193906f542d09c71abb3c020118bba not found: ID does not exist" Jul 09 13:59:59.300472 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:59.300383 2573 scope.go:117] "RemoveContainer" containerID="4426c674ed95e65e8480b4dca698df9f46c2115a1b3a1eb836df8ae4546b66f1" Jul 09 13:59:59.300679 ip-10-0-142-87 kubenswrapper[2573]: E0709 13:59:59.300652 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4426c674ed95e65e8480b4dca698df9f46c2115a1b3a1eb836df8ae4546b66f1\": container with ID starting with 4426c674ed95e65e8480b4dca698df9f46c2115a1b3a1eb836df8ae4546b66f1 not found: ID does not exist" containerID="4426c674ed95e65e8480b4dca698df9f46c2115a1b3a1eb836df8ae4546b66f1" Jul 09 13:59:59.300751 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:59.300676 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4426c674ed95e65e8480b4dca698df9f46c2115a1b3a1eb836df8ae4546b66f1"} err="failed to get container status \"4426c674ed95e65e8480b4dca698df9f46c2115a1b3a1eb836df8ae4546b66f1\": rpc error: code = NotFound desc = could not find container \"4426c674ed95e65e8480b4dca698df9f46c2115a1b3a1eb836df8ae4546b66f1\": container with ID starting with 4426c674ed95e65e8480b4dca698df9f46c2115a1b3a1eb836df8ae4546b66f1 not found: ID does not exist" Jul 09 13:59:59.301925 ip-10-0-142-87 kubenswrapper[2573]: I0709 13:59:59.301907 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-d7d5695b-openshift-default-7c7f998c4c-mmkbg"] Jul 09 14:00:00.151931 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:00:00.151892 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726760-k2xvl"] Jul 09 14:00:00.152364 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:00:00.152296 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3af93efd-b94f-4d10-a648-c4ae04d61115" containerName="istio-proxy" Jul 09 14:00:00.152364 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:00:00.152308 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3af93efd-b94f-4d10-a648-c4ae04d61115" containerName="istio-proxy" Jul 09 14:00:00.152364 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:00:00.152317 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3af93efd-b94f-4d10-a648-c4ae04d61115" containerName="istio-proxy" Jul 09 14:00:00.152364 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:00:00.152323 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3af93efd-b94f-4d10-a648-c4ae04d61115" containerName="istio-proxy" Jul 09 14:00:00.152519 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:00:00.152382 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3af93efd-b94f-4d10-a648-c4ae04d61115" containerName="istio-proxy" Jul 09 14:00:00.157318 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:00:00.157299 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726760-k2xvl" Jul 09 14:00:00.160108 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:00:00.160084 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 09 14:00:00.161255 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:00:00.161228 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-z2qnx\"" Jul 09 14:00:00.161397 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:00:00.161242 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 09 14:00:00.166511 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:00:00.166484 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726760-k2xvl"] Jul 09 14:00:00.282154 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:00:00.282106 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6nng\" (UniqueName: \"kubernetes.io/projected/114ceee0-ba70-4222-b856-decad06f027f-kube-api-access-w6nng\") pod \"maas-api-key-cleanup-29726760-k2xvl\" (UID: \"114ceee0-ba70-4222-b856-decad06f027f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726760-k2xvl" Jul 09 14:00:00.383134 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:00:00.383089 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6nng\" (UniqueName: \"kubernetes.io/projected/114ceee0-ba70-4222-b856-decad06f027f-kube-api-access-w6nng\") pod \"maas-api-key-cleanup-29726760-k2xvl\" (UID: \"114ceee0-ba70-4222-b856-decad06f027f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726760-k2xvl" Jul 09 14:00:00.396913 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:00:00.396873 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6nng\" (UniqueName: \"kubernetes.io/projected/114ceee0-ba70-4222-b856-decad06f027f-kube-api-access-w6nng\") pod \"maas-api-key-cleanup-29726760-k2xvl\" (UID: \"114ceee0-ba70-4222-b856-decad06f027f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726760-k2xvl" Jul 09 14:00:00.471409 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:00:00.471317 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726760-k2xvl" Jul 09 14:00:00.612888 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:00:00.612741 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726760-k2xvl"] Jul 09 14:00:00.615848 ip-10-0-142-87 kubenswrapper[2573]: W0709 14:00:00.615820 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod114ceee0_ba70_4222_b856_decad06f027f.slice/crio-2313a84570eac4c02351981ec20215637fa5621a87f1a7228369b1d12b18de0f WatchSource:0}: Error finding container 2313a84570eac4c02351981ec20215637fa5621a87f1a7228369b1d12b18de0f: Status 404 returned error can't find the container with id 2313a84570eac4c02351981ec20215637fa5621a87f1a7228369b1d12b18de0f Jul 09 14:00:00.946748 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:00:00.946700 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3af93efd-b94f-4d10-a648-c4ae04d61115" path="/var/lib/kubelet/pods/3af93efd-b94f-4d10-a648-c4ae04d61115/volumes" Jul 09 14:00:01.291884 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:00:01.291848 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726760-k2xvl" event={"ID":"114ceee0-ba70-4222-b856-decad06f027f","Type":"ContainerStarted","Data":"d32eec8a0f90a53fa27081f867f36bda0b9a40c0c84c9c32aa44832f386bcd8f"} Jul 09 14:00:01.292308 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:00:01.291911 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726760-k2xvl" event={"ID":"114ceee0-ba70-4222-b856-decad06f027f","Type":"ContainerStarted","Data":"2313a84570eac4c02351981ec20215637fa5621a87f1a7228369b1d12b18de0f"} Jul 09 14:00:01.317434 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:00:01.317377 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726760-k2xvl" podStartSLOduration=1.317359218 podStartE2EDuration="1.317359218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:01.316195068 +0000 UTC m=+1818.821497337" watchObservedRunningTime="2026-07-09 14:00:01.317359218 +0000 UTC m=+1818.822661497" Jul 09 14:00:04.487841 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:00:04.487802 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:00:12.882995 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:00:12.882940 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:01:00.012375 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:01:00.012330 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726760-k2xvl"] Jul 09 14:01:00.012952 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:01:00.012630 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726760-k2xvl" podUID="114ceee0-ba70-4222-b856-decad06f027f" containerName="cleanup" containerID="cri-o://d32eec8a0f90a53fa27081f867f36bda0b9a40c0c84c9c32aa44832f386bcd8f" gracePeriod=30 Jul 09 14:01:24.777496 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:01:24.777457 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:01:30.198897 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:01:30.198867 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726760-k2xvl" Jul 09 14:01:30.372319 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:01:30.372215 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6nng\" (UniqueName: \"kubernetes.io/projected/114ceee0-ba70-4222-b856-decad06f027f-kube-api-access-w6nng\") pod \"114ceee0-ba70-4222-b856-decad06f027f\" (UID: \"114ceee0-ba70-4222-b856-decad06f027f\") " Jul 09 14:01:30.374406 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:01:30.374374 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/114ceee0-ba70-4222-b856-decad06f027f-kube-api-access-w6nng" (OuterVolumeSpecName: "kube-api-access-w6nng") pod "114ceee0-ba70-4222-b856-decad06f027f" (UID: "114ceee0-ba70-4222-b856-decad06f027f"). InnerVolumeSpecName "kube-api-access-w6nng". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:01:30.473945 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:01:30.473902 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6nng\" (UniqueName: \"kubernetes.io/projected/114ceee0-ba70-4222-b856-decad06f027f-kube-api-access-w6nng\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:01:30.761463 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:01:30.761425 2573 generic.go:356] "Generic (PLEG): container finished" podID="114ceee0-ba70-4222-b856-decad06f027f" containerID="d32eec8a0f90a53fa27081f867f36bda0b9a40c0c84c9c32aa44832f386bcd8f" exitCode=137 Jul 09 14:01:30.761463 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:01:30.761467 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726760-k2xvl" Jul 09 14:01:30.761832 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:01:30.761475 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726760-k2xvl" event={"ID":"114ceee0-ba70-4222-b856-decad06f027f","Type":"ContainerDied","Data":"d32eec8a0f90a53fa27081f867f36bda0b9a40c0c84c9c32aa44832f386bcd8f"} Jul 09 14:01:30.761832 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:01:30.761526 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726760-k2xvl" event={"ID":"114ceee0-ba70-4222-b856-decad06f027f","Type":"ContainerDied","Data":"2313a84570eac4c02351981ec20215637fa5621a87f1a7228369b1d12b18de0f"} Jul 09 14:01:30.761832 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:01:30.761554 2573 scope.go:117] "RemoveContainer" containerID="d32eec8a0f90a53fa27081f867f36bda0b9a40c0c84c9c32aa44832f386bcd8f" Jul 09 14:01:30.770949 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:01:30.770924 2573 scope.go:117] "RemoveContainer" containerID="d32eec8a0f90a53fa27081f867f36bda0b9a40c0c84c9c32aa44832f386bcd8f" Jul 09 14:01:30.771292 ip-10-0-142-87 kubenswrapper[2573]: E0709 14:01:30.771272 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d32eec8a0f90a53fa27081f867f36bda0b9a40c0c84c9c32aa44832f386bcd8f\": container with ID starting with d32eec8a0f90a53fa27081f867f36bda0b9a40c0c84c9c32aa44832f386bcd8f not found: ID does not exist" containerID="d32eec8a0f90a53fa27081f867f36bda0b9a40c0c84c9c32aa44832f386bcd8f" Jul 09 14:01:30.771340 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:01:30.771302 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d32eec8a0f90a53fa27081f867f36bda0b9a40c0c84c9c32aa44832f386bcd8f"} err="failed to get container status \"d32eec8a0f90a53fa27081f867f36bda0b9a40c0c84c9c32aa44832f386bcd8f\": rpc error: code = NotFound desc = could not find container \"d32eec8a0f90a53fa27081f867f36bda0b9a40c0c84c9c32aa44832f386bcd8f\": container with ID starting with d32eec8a0f90a53fa27081f867f36bda0b9a40c0c84c9c32aa44832f386bcd8f not found: ID does not exist" Jul 09 14:01:30.791332 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:01:30.791299 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726760-k2xvl"] Jul 09 14:01:30.799171 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:01:30.799142 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726760-k2xvl"] Jul 09 14:01:30.946080 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:01:30.946047 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="114ceee0-ba70-4222-b856-decad06f027f" path="/var/lib/kubelet/pods/114ceee0-ba70-4222-b856-decad06f027f/volumes" Jul 09 14:02:02.335057 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.332545 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd"] Jul 09 14:02:02.335057 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.333542 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="114ceee0-ba70-4222-b856-decad06f027f" containerName="cleanup" Jul 09 14:02:02.335057 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.333570 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="114ceee0-ba70-4222-b856-decad06f027f" containerName="cleanup" Jul 09 14:02:02.335057 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.333744 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3af93efd-b94f-4d10-a648-c4ae04d61115" containerName="istio-proxy" Jul 09 14:02:02.335057 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.333769 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="114ceee0-ba70-4222-b856-decad06f027f" containerName="cleanup" Jul 09 14:02:02.338949 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.338918 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.341899 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.341867 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-440311-openshift-default-dockercfg-zmjw2\"" Jul 09 14:02:02.345799 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.345770 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd"] Jul 09 14:02:02.497675 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.497632 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-workload-socket\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.497884 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.497683 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-credential-socket\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.497884 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.497706 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f42268e9-719a-4a80-bf65-134f5087e17f-istio-token\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.497884 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.497808 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-workload-certs\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.497884 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.497846 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q68w\" (UniqueName: \"kubernetes.io/projected/f42268e9-719a-4a80-bf65-134f5087e17f-kube-api-access-9q68w\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.498100 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.497939 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-istio-envoy\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.498100 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.497956 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-istio-data\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.498100 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.497981 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f42268e9-719a-4a80-bf65-134f5087e17f-istio-podinfo\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.498100 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.497996 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f42268e9-719a-4a80-bf65-134f5087e17f-istiod-ca-cert\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.599109 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.598960 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-workload-certs\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.599109 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.599010 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q68w\" (UniqueName: \"kubernetes.io/projected/f42268e9-719a-4a80-bf65-134f5087e17f-kube-api-access-9q68w\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.599353 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.599118 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-istio-envoy\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.599353 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.599145 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-istio-data\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.599353 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.599181 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f42268e9-719a-4a80-bf65-134f5087e17f-istio-podinfo\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.599353 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.599204 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f42268e9-719a-4a80-bf65-134f5087e17f-istiod-ca-cert\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.599353 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.599268 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-workload-socket\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.599353 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.599309 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-credential-socket\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.599353 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.599340 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f42268e9-719a-4a80-bf65-134f5087e17f-istio-token\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.599716 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.599493 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-workload-certs\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.599716 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.599582 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-istio-data\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.599788 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.599740 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-credential-socket\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.599824 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.599806 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-workload-socket\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.600135 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.600109 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f42268e9-719a-4a80-bf65-134f5087e17f-istiod-ca-cert\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.601594 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.601575 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-istio-envoy\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.601807 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.601790 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f42268e9-719a-4a80-bf65-134f5087e17f-istio-podinfo\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.608823 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.608790 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f42268e9-719a-4a80-bf65-134f5087e17f-istio-token\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.608823 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.608807 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q68w\" (UniqueName: \"kubernetes.io/projected/f42268e9-719a-4a80-bf65-134f5087e17f-kube-api-access-9q68w\") pod \"e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.652714 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.652668 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:02.799923 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.799889 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd"] Jul 09 14:02:02.802179 ip-10-0-142-87 kubenswrapper[2573]: W0709 14:02:02.802155 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf42268e9_719a_4a80_bf65_134f5087e17f.slice/crio-bc43139f8cfe0862cbd09ae351d2983266e9f1456302651265bd98ec4ea3fd9f WatchSource:0}: Error finding container bc43139f8cfe0862cbd09ae351d2983266e9f1456302651265bd98ec4ea3fd9f: Status 404 returned error can't find the container with id bc43139f8cfe0862cbd09ae351d2983266e9f1456302651265bd98ec4ea3fd9f Jul 09 14:02:02.804787 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.804752 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:02:02.804886 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.804822 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:02:02.804886 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.804869 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:02:02.930374 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.930334 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" event={"ID":"f42268e9-719a-4a80-bf65-134f5087e17f","Type":"ContainerStarted","Data":"8b912bb855db51f89b563db4f398e18ed8ef08dcd08006d588fc2a27bd3890e8"} Jul 09 14:02:02.930562 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.930395 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" event={"ID":"f42268e9-719a-4a80-bf65-134f5087e17f","Type":"ContainerStarted","Data":"bc43139f8cfe0862cbd09ae351d2983266e9f1456302651265bd98ec4ea3fd9f"} Jul 09 14:02:02.953203 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:02.953137 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" podStartSLOduration=0.953115422 podStartE2EDuration="953.115422ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:02:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:02:02.950849548 +0000 UTC m=+1940.456151851" watchObservedRunningTime="2026-07-09 14:02:02.953115422 +0000 UTC m=+1940.458417691" Jul 09 14:02:03.653616 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:03.653583 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:03.658476 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:03.658448 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:03.935567 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:03.935476 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:03.936533 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:03.936511 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:10.011086 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.011049 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n"] Jul 09 14:02:10.014746 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.014720 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.017275 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.017251 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-505c59-openshift-default-dockercfg-d769w\"" Jul 09 14:02:10.027197 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.027171 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n"] Jul 09 14:02:10.075782 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.075747 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9663bf3a-fae2-4cc9-aa82-166afd101bda-istio-podinfo\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.075782 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.075785 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-workload-socket\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.076006 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.075812 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9663bf3a-fae2-4cc9-aa82-166afd101bda-istio-token\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.076006 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.075885 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrn6l\" (UniqueName: \"kubernetes.io/projected/9663bf3a-fae2-4cc9-aa82-166afd101bda-kube-api-access-rrn6l\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.076006 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.075977 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-credential-socket\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.076150 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.076075 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-istio-envoy\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.076150 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.076119 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-workload-certs\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.076220 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.076181 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-istio-data\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.076220 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.076199 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9663bf3a-fae2-4cc9-aa82-166afd101bda-istiod-ca-cert\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.177306 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.177267 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-istio-data\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.177306 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.177307 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9663bf3a-fae2-4cc9-aa82-166afd101bda-istiod-ca-cert\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.177568 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.177344 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9663bf3a-fae2-4cc9-aa82-166afd101bda-istio-podinfo\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.177568 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.177376 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-workload-socket\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.177568 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.177405 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9663bf3a-fae2-4cc9-aa82-166afd101bda-istio-token\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.177568 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.177423 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrn6l\" (UniqueName: \"kubernetes.io/projected/9663bf3a-fae2-4cc9-aa82-166afd101bda-kube-api-access-rrn6l\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.177568 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.177470 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-credential-socket\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.177568 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.177524 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-istio-envoy\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.177863 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.177571 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-workload-certs\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.178149 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.178109 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-workload-socket\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.178327 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.178303 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-credential-socket\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.178414 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.178388 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-istio-data\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.180926 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.178962 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9663bf3a-fae2-4cc9-aa82-166afd101bda-istiod-ca-cert\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.180926 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.179005 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-workload-certs\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.181123 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.181059 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9663bf3a-fae2-4cc9-aa82-166afd101bda-istio-podinfo\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.181123 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.181112 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-istio-envoy\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.187982 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.187951 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9663bf3a-fae2-4cc9-aa82-166afd101bda-istio-token\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.189269 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.189251 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrn6l\" (UniqueName: \"kubernetes.io/projected/9663bf3a-fae2-4cc9-aa82-166afd101bda-kube-api-access-rrn6l\") pod \"e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.327620 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.327531 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:10.475934 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.475907 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n"] Jul 09 14:02:10.478110 ip-10-0-142-87 kubenswrapper[2573]: W0709 14:02:10.478068 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9663bf3a_fae2_4cc9_aa82_166afd101bda.slice/crio-ec401b6b32cfaada91cdfe60cf999b5c0cf30adcc6a200e0c1204791b9c4cbfd WatchSource:0}: Error finding container ec401b6b32cfaada91cdfe60cf999b5c0cf30adcc6a200e0c1204791b9c4cbfd: Status 404 returned error can't find the container with id ec401b6b32cfaada91cdfe60cf999b5c0cf30adcc6a200e0c1204791b9c4cbfd Jul 09 14:02:10.480282 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.480244 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:02:10.480397 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.480329 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:02:10.480469 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.480399 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:02:10.976885 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.976846 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" event={"ID":"9663bf3a-fae2-4cc9-aa82-166afd101bda","Type":"ContainerStarted","Data":"deadf063bde5c31f8ef06fbc4c6bce71f67a2ac5c16ba06bd646a80cee7384ab"} Jul 09 14:02:10.977124 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.976907 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" event={"ID":"9663bf3a-fae2-4cc9-aa82-166afd101bda","Type":"ContainerStarted","Data":"ec401b6b32cfaada91cdfe60cf999b5c0cf30adcc6a200e0c1204791b9c4cbfd"} Jul 09 14:02:10.998389 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:10.998342 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" podStartSLOduration=1.9983245699999999 podStartE2EDuration="1.99832457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:02:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:02:10.997176885 +0000 UTC m=+1948.502479154" watchObservedRunningTime="2026-07-09 14:02:10.99832457 +0000 UTC m=+1948.503626837" Jul 09 14:02:11.328796 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:11.328698 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:11.333610 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:11.333584 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:11.981921 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:11.981881 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:11.982949 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:11.982929 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:12.475807 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:12.475764 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:02:19.977250 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:19.977194 2573 patch_prober.go:28] interesting pod/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 14:02:19.977764 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:19.977284 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" podUID="f42268e9-719a-4a80-bf65-134f5087e17f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 14:02:22.038930 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:22.038896 2573 generic.go:356] "Generic (PLEG): container finished" podID="f42268e9-719a-4a80-bf65-134f5087e17f" containerID="8b912bb855db51f89b563db4f398e18ed8ef08dcd08006d588fc2a27bd3890e8" exitCode=0 Jul 09 14:02:22.039415 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:22.038970 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" event={"ID":"f42268e9-719a-4a80-bf65-134f5087e17f","Type":"ContainerDied","Data":"8b912bb855db51f89b563db4f398e18ed8ef08dcd08006d588fc2a27bd3890e8"} Jul 09 14:02:22.039415 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:22.039402 2573 scope.go:117] "RemoveContainer" containerID="8b912bb855db51f89b563db4f398e18ed8ef08dcd08006d588fc2a27bd3890e8" Jul 09 14:02:22.040425 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:22.040397 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:02:22.040478 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:22.040466 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:02:22.040513 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:22.040498 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:02:23.046161 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:23.046121 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" event={"ID":"f42268e9-719a-4a80-bf65-134f5087e17f","Type":"ContainerStarted","Data":"16f519362d6d79367e3bfb1d9a50294dd9e9fa4aa9ba04d7c0fa0201a816c4f3"} Jul 09 14:02:23.653118 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:23.653076 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:23.653374 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:23.653348 2573 patch_prober.go:28] interesting pod/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 09 14:02:23.653430 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:23.653413 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" podUID="f42268e9-719a-4a80-bf65-134f5087e17f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 09 14:02:24.476385 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:24.476343 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:02:24.653542 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:24.653501 2573 patch_prober.go:28] interesting pod/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 09 14:02:24.653717 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:24.653570 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" podUID="f42268e9-719a-4a80-bf65-134f5087e17f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 09 14:02:25.653107 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:25.653069 2573 patch_prober.go:28] interesting pod/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 09 14:02:25.653569 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:25.653141 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" podUID="f42268e9-719a-4a80-bf65-134f5087e17f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 09 14:02:26.653938 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:26.653887 2573 patch_prober.go:28] interesting pod/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 09 14:02:26.654496 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:26.653954 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" podUID="f42268e9-719a-4a80-bf65-134f5087e17f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 09 14:02:27.653872 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:27.653828 2573 patch_prober.go:28] interesting pod/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 09 14:02:27.654086 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:27.653900 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" podUID="f42268e9-719a-4a80-bf65-134f5087e17f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 09 14:02:27.941912 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:27.941878 2573 patch_prober.go:28] interesting pod/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": read tcp 10.134.0.2:55220->10.134.0.43:15021: read: connection reset by peer" start-of-body= Jul 09 14:02:27.942065 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:27.941949 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" podUID="9663bf3a-fae2-4cc9-aa82-166afd101bda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": read tcp 10.134.0.2:55220->10.134.0.43:15021: read: connection reset by peer" Jul 09 14:02:28.074666 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:28.074631 2573 generic.go:356] "Generic (PLEG): container finished" podID="9663bf3a-fae2-4cc9-aa82-166afd101bda" containerID="deadf063bde5c31f8ef06fbc4c6bce71f67a2ac5c16ba06bd646a80cee7384ab" exitCode=0 Jul 09 14:02:28.074865 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:28.074707 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" event={"ID":"9663bf3a-fae2-4cc9-aa82-166afd101bda","Type":"ContainerDied","Data":"deadf063bde5c31f8ef06fbc4c6bce71f67a2ac5c16ba06bd646a80cee7384ab"} Jul 09 14:02:28.075251 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:28.075233 2573 scope.go:117] "RemoveContainer" containerID="deadf063bde5c31f8ef06fbc4c6bce71f67a2ac5c16ba06bd646a80cee7384ab" Jul 09 14:02:28.076298 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:28.076268 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:02:28.076400 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:28.076356 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:02:28.076444 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:28.076420 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:02:28.653215 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:28.653179 2573 patch_prober.go:28] interesting pod/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 09 14:02:28.653399 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:28.653233 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" podUID="f42268e9-719a-4a80-bf65-134f5087e17f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 09 14:02:29.081968 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:29.081858 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" event={"ID":"9663bf3a-fae2-4cc9-aa82-166afd101bda","Type":"ContainerStarted","Data":"8cfcb5e2192dc88f47491442bb3618f59e9f771d997055571d1ce04c0456eea6"} Jul 09 14:02:29.328131 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:29.328083 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:29.328347 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:29.328278 2573 patch_prober.go:28] interesting pod/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 09 14:02:29.328347 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:29.328330 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" podUID="9663bf3a-fae2-4cc9-aa82-166afd101bda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 09 14:02:29.656508 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:29.656475 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:29.656777 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:29.656760 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:29.657507 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:29.657483 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:02:30.328417 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:30.328376 2573 patch_prober.go:28] interesting pod/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 09 14:02:30.328818 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:30.328447 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" podUID="9663bf3a-fae2-4cc9-aa82-166afd101bda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 09 14:02:31.328870 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:31.328829 2573 patch_prober.go:28] interesting pod/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 09 14:02:31.329298 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:31.328914 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" podUID="9663bf3a-fae2-4cc9-aa82-166afd101bda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 09 14:02:32.328627 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:32.328585 2573 patch_prober.go:28] interesting pod/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 09 14:02:32.328822 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:32.328645 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" podUID="9663bf3a-fae2-4cc9-aa82-166afd101bda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 09 14:02:33.328792 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:33.328751 2573 patch_prober.go:28] interesting pod/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 09 14:02:33.329343 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:33.328812 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" podUID="9663bf3a-fae2-4cc9-aa82-166afd101bda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 09 14:02:34.332107 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:34.332011 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:34.332496 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:34.332408 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:34.333161 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:34.333143 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:02:37.070953 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:37.070920 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:02:58.468039 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:02:58.467983 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:03:15.660397 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.660362 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm"] Jul 09 14:03:15.663962 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.663944 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.666552 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.666526 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-reuse-d43b1d-openshift-default-dockercfg-5v54g\"" Jul 09 14:03:15.680650 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.680617 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm"] Jul 09 14:03:15.780558 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.780520 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec10af80-f607-4ce3-a666-85a31996898e-istio-token\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.780558 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.780560 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-credential-socket\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.780824 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.780580 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-istio-envoy\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.780824 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.780602 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqdkv\" (UniqueName: \"kubernetes.io/projected/ec10af80-f607-4ce3-a666-85a31996898e-kube-api-access-mqdkv\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.780824 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.780622 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-workload-socket\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.780824 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.780689 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-istio-data\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.780824 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.780738 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec10af80-f607-4ce3-a666-85a31996898e-istiod-ca-cert\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.780824 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.780806 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-workload-certs\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.781128 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.780919 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec10af80-f607-4ce3-a666-85a31996898e-istio-podinfo\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.882470 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.882429 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec10af80-f607-4ce3-a666-85a31996898e-istiod-ca-cert\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.882470 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.882472 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-workload-certs\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.882753 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.882528 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec10af80-f607-4ce3-a666-85a31996898e-istio-podinfo\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.882753 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.882567 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec10af80-f607-4ce3-a666-85a31996898e-istio-token\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.882753 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.882585 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-credential-socket\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.882753 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.882602 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-istio-envoy\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.882753 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.882632 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqdkv\" (UniqueName: \"kubernetes.io/projected/ec10af80-f607-4ce3-a666-85a31996898e-kube-api-access-mqdkv\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.882753 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.882658 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-workload-socket\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.882753 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.882699 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-istio-data\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.883302 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.882995 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-credential-socket\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.883302 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.883073 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-workload-certs\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.883302 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.883229 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-istio-data\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.883437 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.883339 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec10af80-f607-4ce3-a666-85a31996898e-istiod-ca-cert\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.883437 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.883348 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-workload-socket\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.885078 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.885055 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-istio-envoy\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.885278 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.885264 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec10af80-f607-4ce3-a666-85a31996898e-istio-podinfo\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.891611 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.891583 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqdkv\" (UniqueName: \"kubernetes.io/projected/ec10af80-f607-4ce3-a666-85a31996898e-kube-api-access-mqdkv\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.891732 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.891654 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec10af80-f607-4ce3-a666-85a31996898e-istio-token\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:15.974664 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:15.974571 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:16.122291 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:16.122192 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm"] Jul 09 14:03:16.124786 ip-10-0-142-87 kubenswrapper[2573]: W0709 14:03:16.124747 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec10af80_f607_4ce3_a666_85a31996898e.slice/crio-3e0215baff070b59a9616cc79440349298f7fe09c64dba2e0d02f86a1f508b5e WatchSource:0}: Error finding container 3e0215baff070b59a9616cc79440349298f7fe09c64dba2e0d02f86a1f508b5e: Status 404 returned error can't find the container with id 3e0215baff070b59a9616cc79440349298f7fe09c64dba2e0d02f86a1f508b5e Jul 09 14:03:16.126795 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:16.126759 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:03:16.126881 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:16.126830 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:03:16.126881 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:16.126859 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:03:16.348393 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:16.348301 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" event={"ID":"ec10af80-f607-4ce3-a666-85a31996898e","Type":"ContainerStarted","Data":"88fdea9a465751b35cb9048d18ef9d9308b07357f37ad42dd324184af7dd2693"} Jul 09 14:03:16.348393 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:16.348367 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" event={"ID":"ec10af80-f607-4ce3-a666-85a31996898e","Type":"ContainerStarted","Data":"3e0215baff070b59a9616cc79440349298f7fe09c64dba2e0d02f86a1f508b5e"} Jul 09 14:03:16.373605 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:16.373547 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" podStartSLOduration=1.373523683 podStartE2EDuration="1.373523683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:03:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:03:16.371627671 +0000 UTC m=+2013.876929942" watchObservedRunningTime="2026-07-09 14:03:16.373523683 +0000 UTC m=+2013.878825951" Jul 09 14:03:16.975043 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:16.974979 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:17.980593 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:17.980551 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:18.359460 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:18.359366 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:18.360365 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:18.360343 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:23.579584 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:23.579543 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:03:32.439689 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:32.439651 2573 generic.go:356] "Generic (PLEG): container finished" podID="ec10af80-f607-4ce3-a666-85a31996898e" containerID="88fdea9a465751b35cb9048d18ef9d9308b07357f37ad42dd324184af7dd2693" exitCode=0 Jul 09 14:03:32.440152 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:32.439724 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" event={"ID":"ec10af80-f607-4ce3-a666-85a31996898e","Type":"ContainerDied","Data":"88fdea9a465751b35cb9048d18ef9d9308b07357f37ad42dd324184af7dd2693"} Jul 09 14:03:32.440200 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:32.440170 2573 scope.go:117] "RemoveContainer" containerID="88fdea9a465751b35cb9048d18ef9d9308b07357f37ad42dd324184af7dd2693" Jul 09 14:03:32.441238 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:32.441208 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:03:32.441352 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:32.441293 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:03:32.441352 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:32.441340 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:03:33.360161 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:33.360119 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:33.447414 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:33.447366 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" event={"ID":"ec10af80-f607-4ce3-a666-85a31996898e","Type":"ContainerStarted","Data":"6dd450fd202167b42fcac2b9a538364217219e1a02e179a8f5a2496e2612795f"} Jul 09 14:03:33.975133 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:33.975091 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:33.976418 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:33.976390 2573 patch_prober.go:28] interesting pod/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 09 14:03:33.976550 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:33.976457 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" podUID="ec10af80-f607-4ce3-a666-85a31996898e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 09 14:03:34.975473 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:34.975437 2573 patch_prober.go:28] interesting pod/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 09 14:03:34.975874 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:34.975495 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" podUID="ec10af80-f607-4ce3-a666-85a31996898e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 09 14:03:35.975094 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:35.975055 2573 patch_prober.go:28] interesting pod/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 09 14:03:35.975094 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:35.975111 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" podUID="ec10af80-f607-4ce3-a666-85a31996898e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 09 14:03:36.975464 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:36.975426 2573 patch_prober.go:28] interesting pod/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 09 14:03:36.975881 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:36.975486 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" podUID="ec10af80-f607-4ce3-a666-85a31996898e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 09 14:03:37.979282 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:37.979250 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:37.979752 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:37.979552 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:37.980199 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:37.980182 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:03:53.870419 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:03:53.870380 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:04:31.071949 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:31.071910 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:04:34.384790 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:34.384755 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm"] Jul 09 14:04:34.385295 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:34.384994 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" podUID="ec10af80-f607-4ce3-a666-85a31996898e" containerName="istio-proxy" containerID="cri-o://6dd450fd202167b42fcac2b9a538364217219e1a02e179a8f5a2496e2612795f" gracePeriod=30 Jul 09 14:04:35.283212 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.283172 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m"] Jul 09 14:04:35.286900 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.286877 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.289729 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.289703 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-reuse-d43b1d-openshift-default-dockercfg-smlxh\"" Jul 09 14:04:35.300382 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.300352 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m"] Jul 09 14:04:35.395247 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.395204 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istiod-ca-cert\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.395789 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.395760 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istio-envoy\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.395943 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.395809 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-credential-socket\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.395943 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.395847 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-workload-socket\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.395943 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.395906 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istio-data\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.396149 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.395944 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istio-podinfo\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.396149 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.395968 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istio-token\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.396149 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.396081 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs442\" (UniqueName: \"kubernetes.io/projected/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-kube-api-access-vs442\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.396149 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.396128 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-workload-certs\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.497217 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.497175 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs442\" (UniqueName: \"kubernetes.io/projected/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-kube-api-access-vs442\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.497391 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.497234 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-workload-certs\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.497391 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.497285 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istiod-ca-cert\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.497391 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.497305 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istio-envoy\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.497391 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.497337 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-credential-socket\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.497391 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.497369 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-workload-socket\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.497666 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.497418 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istio-data\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.497666 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.497443 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istio-podinfo\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.497666 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.497546 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istio-token\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.497830 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.497692 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-workload-certs\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.497889 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.497832 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-credential-socket\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.498003 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.497980 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istio-data\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.498152 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.498006 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-workload-socket\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.498222 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.498165 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istiod-ca-cert\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.499777 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.499753 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istio-envoy\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.499950 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.499933 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istio-podinfo\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.509093 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.509034 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istio-token\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.509235 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.509179 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs442\" (UniqueName: \"kubernetes.io/projected/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-kube-api-access-vs442\") pod \"e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.598656 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.598547 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:35.747801 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.747701 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m"] Jul 09 14:04:35.750685 ip-10-0-142-87 kubenswrapper[2573]: W0709 14:04:35.750655 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c7d3cc8_84b2_40c0_a5ac_14fee9eff994.slice/crio-8d5a0be32a7b6b40280f80148a5d4740a4edd7372036dc52d5318977970aa328 WatchSource:0}: Error finding container 8d5a0be32a7b6b40280f80148a5d4740a4edd7372036dc52d5318977970aa328: Status 404 returned error can't find the container with id 8d5a0be32a7b6b40280f80148a5d4740a4edd7372036dc52d5318977970aa328 Jul 09 14:04:35.752918 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.752882 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:04:35.752994 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.752948 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:04:35.752994 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.752983 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:04:35.770927 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:35.770900 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" event={"ID":"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994","Type":"ContainerStarted","Data":"8d5a0be32a7b6b40280f80148a5d4740a4edd7372036dc52d5318977970aa328"} Jul 09 14:04:36.778126 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:36.778080 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" event={"ID":"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994","Type":"ContainerStarted","Data":"39809398ad9ea4dde20d973073825fd2d341393b27623e7ae9fec04f0a265447"} Jul 09 14:04:36.800969 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:36.800900 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" podStartSLOduration=1.8008834249999999 podStartE2EDuration="1.800883425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:04:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:04:36.799558307 +0000 UTC m=+2094.304860599" watchObservedRunningTime="2026-07-09 14:04:36.800883425 +0000 UTC m=+2094.306185692" Jul 09 14:04:37.599111 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:37.599069 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:37.604002 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:37.603973 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:37.784935 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:37.784899 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:37.785967 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:37.785941 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:37.981738 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:37.981697 2573 patch_prober.go:28] interesting pod/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 14:04:37.981930 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:37.981771 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" podUID="ec10af80-f607-4ce3-a666-85a31996898e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 14:04:39.650894 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.650822 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:04:39.743054 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.742989 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-istio-envoy\") pod \"ec10af80-f607-4ce3-a666-85a31996898e\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " Jul 09 14:04:39.743242 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.743069 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-workload-certs\") pod \"ec10af80-f607-4ce3-a666-85a31996898e\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " Jul 09 14:04:39.743242 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.743113 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-credential-socket\") pod \"ec10af80-f607-4ce3-a666-85a31996898e\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " Jul 09 14:04:39.743242 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.743149 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqdkv\" (UniqueName: \"kubernetes.io/projected/ec10af80-f607-4ce3-a666-85a31996898e-kube-api-access-mqdkv\") pod \"ec10af80-f607-4ce3-a666-85a31996898e\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " Jul 09 14:04:39.743242 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.743214 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-istio-data\") pod \"ec10af80-f607-4ce3-a666-85a31996898e\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " Jul 09 14:04:39.743470 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.743254 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec10af80-f607-4ce3-a666-85a31996898e-istio-podinfo\") pod \"ec10af80-f607-4ce3-a666-85a31996898e\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " Jul 09 14:04:39.743470 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.743306 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-workload-socket\") pod \"ec10af80-f607-4ce3-a666-85a31996898e\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " Jul 09 14:04:39.743470 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.743364 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec10af80-f607-4ce3-a666-85a31996898e-istiod-ca-cert\") pod \"ec10af80-f607-4ce3-a666-85a31996898e\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " Jul 09 14:04:39.743470 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.743392 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec10af80-f607-4ce3-a666-85a31996898e-istio-token\") pod \"ec10af80-f607-4ce3-a666-85a31996898e\" (UID: \"ec10af80-f607-4ce3-a666-85a31996898e\") " Jul 09 14:04:39.743855 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.743385 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ec10af80-f607-4ce3-a666-85a31996898e" (UID: "ec10af80-f607-4ce3-a666-85a31996898e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:04:39.743855 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.743399 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ec10af80-f607-4ce3-a666-85a31996898e" (UID: "ec10af80-f607-4ce3-a666-85a31996898e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:04:39.744079 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.743778 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec10af80-f607-4ce3-a666-85a31996898e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ec10af80-f607-4ce3-a666-85a31996898e" (UID: "ec10af80-f607-4ce3-a666-85a31996898e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:04:39.744079 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.743784 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ec10af80-f607-4ce3-a666-85a31996898e" (UID: "ec10af80-f607-4ce3-a666-85a31996898e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:04:39.744207 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.744159 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-istio-data" (OuterVolumeSpecName: "istio-data") pod "ec10af80-f607-4ce3-a666-85a31996898e" (UID: "ec10af80-f607-4ce3-a666-85a31996898e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:04:39.745455 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.745425 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ec10af80-f607-4ce3-a666-85a31996898e" (UID: "ec10af80-f607-4ce3-a666-85a31996898e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:04:39.745589 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.745564 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ec10af80-f607-4ce3-a666-85a31996898e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ec10af80-f607-4ce3-a666-85a31996898e" (UID: "ec10af80-f607-4ce3-a666-85a31996898e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 14:04:39.745770 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.745748 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec10af80-f607-4ce3-a666-85a31996898e-istio-token" (OuterVolumeSpecName: "istio-token") pod "ec10af80-f607-4ce3-a666-85a31996898e" (UID: "ec10af80-f607-4ce3-a666-85a31996898e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:04:39.745874 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.745765 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec10af80-f607-4ce3-a666-85a31996898e-kube-api-access-mqdkv" (OuterVolumeSpecName: "kube-api-access-mqdkv") pod "ec10af80-f607-4ce3-a666-85a31996898e" (UID: "ec10af80-f607-4ce3-a666-85a31996898e"). InnerVolumeSpecName "kube-api-access-mqdkv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:04:39.795672 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.795639 2573 generic.go:356] "Generic (PLEG): container finished" podID="ec10af80-f607-4ce3-a666-85a31996898e" containerID="6dd450fd202167b42fcac2b9a538364217219e1a02e179a8f5a2496e2612795f" exitCode=0 Jul 09 14:04:39.795869 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.795687 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" Jul 09 14:04:39.795869 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.795719 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" event={"ID":"ec10af80-f607-4ce3-a666-85a31996898e","Type":"ContainerDied","Data":"6dd450fd202167b42fcac2b9a538364217219e1a02e179a8f5a2496e2612795f"} Jul 09 14:04:39.795869 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.795772 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm" event={"ID":"ec10af80-f607-4ce3-a666-85a31996898e","Type":"ContainerDied","Data":"3e0215baff070b59a9616cc79440349298f7fe09c64dba2e0d02f86a1f508b5e"} Jul 09 14:04:39.795869 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.795785 2573 scope.go:117] "RemoveContainer" containerID="6dd450fd202167b42fcac2b9a538364217219e1a02e179a8f5a2496e2612795f" Jul 09 14:04:39.805526 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.805502 2573 scope.go:117] "RemoveContainer" containerID="88fdea9a465751b35cb9048d18ef9d9308b07357f37ad42dd324184af7dd2693" Jul 09 14:04:39.814134 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.814108 2573 scope.go:117] "RemoveContainer" containerID="6dd450fd202167b42fcac2b9a538364217219e1a02e179a8f5a2496e2612795f" Jul 09 14:04:39.814459 ip-10-0-142-87 kubenswrapper[2573]: E0709 14:04:39.814437 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6dd450fd202167b42fcac2b9a538364217219e1a02e179a8f5a2496e2612795f\": container with ID starting with 6dd450fd202167b42fcac2b9a538364217219e1a02e179a8f5a2496e2612795f not found: ID does not exist" containerID="6dd450fd202167b42fcac2b9a538364217219e1a02e179a8f5a2496e2612795f" Jul 09 14:04:39.814542 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.814476 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6dd450fd202167b42fcac2b9a538364217219e1a02e179a8f5a2496e2612795f"} err="failed to get container status \"6dd450fd202167b42fcac2b9a538364217219e1a02e179a8f5a2496e2612795f\": rpc error: code = NotFound desc = could not find container \"6dd450fd202167b42fcac2b9a538364217219e1a02e179a8f5a2496e2612795f\": container with ID starting with 6dd450fd202167b42fcac2b9a538364217219e1a02e179a8f5a2496e2612795f not found: ID does not exist" Jul 09 14:04:39.814542 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.814503 2573 scope.go:117] "RemoveContainer" containerID="88fdea9a465751b35cb9048d18ef9d9308b07357f37ad42dd324184af7dd2693" Jul 09 14:04:39.814770 ip-10-0-142-87 kubenswrapper[2573]: E0709 14:04:39.814751 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88fdea9a465751b35cb9048d18ef9d9308b07357f37ad42dd324184af7dd2693\": container with ID starting with 88fdea9a465751b35cb9048d18ef9d9308b07357f37ad42dd324184af7dd2693 not found: ID does not exist" containerID="88fdea9a465751b35cb9048d18ef9d9308b07357f37ad42dd324184af7dd2693" Jul 09 14:04:39.814814 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.814776 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88fdea9a465751b35cb9048d18ef9d9308b07357f37ad42dd324184af7dd2693"} err="failed to get container status \"88fdea9a465751b35cb9048d18ef9d9308b07357f37ad42dd324184af7dd2693\": rpc error: code = NotFound desc = could not find container \"88fdea9a465751b35cb9048d18ef9d9308b07357f37ad42dd324184af7dd2693\": container with ID starting with 88fdea9a465751b35cb9048d18ef9d9308b07357f37ad42dd324184af7dd2693 not found: ID does not exist" Jul 09 14:04:39.819435 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.819404 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm"] Jul 09 14:04:39.824592 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.824557 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-wpbhm"] Jul 09 14:04:39.844940 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.844908 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-credential-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:04:39.844940 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.844936 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqdkv\" (UniqueName: \"kubernetes.io/projected/ec10af80-f607-4ce3-a666-85a31996898e-kube-api-access-mqdkv\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:04:39.845182 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.844950 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-istio-data\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:04:39.845182 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.844962 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec10af80-f607-4ce3-a666-85a31996898e-istio-podinfo\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:04:39.845182 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.844973 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-workload-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:04:39.845182 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.844986 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec10af80-f607-4ce3-a666-85a31996898e-istiod-ca-cert\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:04:39.845182 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.844997 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec10af80-f607-4ce3-a666-85a31996898e-istio-token\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:04:39.845182 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.845007 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-istio-envoy\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:04:39.845182 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:39.845036 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec10af80-f607-4ce3-a666-85a31996898e-workload-certs\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:04:40.946512 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:40.946470 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec10af80-f607-4ce3-a666-85a31996898e" path="/var/lib/kubelet/pods/ec10af80-f607-4ce3-a666-85a31996898e/volumes" Jul 09 14:04:43.035716 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:43.035680 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmms8_3b512d09-0288-4fea-864e-fdc734afb1bd/ovn-acl-logging/0.log" Jul 09 14:04:43.044254 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:43.044229 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmms8_3b512d09-0288-4fea-864e-fdc734afb1bd/ovn-acl-logging/0.log" Jul 09 14:04:51.860889 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:51.860853 2573 generic.go:356] "Generic (PLEG): container finished" podID="1c7d3cc8-84b2-40c0-a5ac-14fee9eff994" containerID="39809398ad9ea4dde20d973073825fd2d341393b27623e7ae9fec04f0a265447" exitCode=0 Jul 09 14:04:51.861428 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:51.860933 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" event={"ID":"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994","Type":"ContainerDied","Data":"39809398ad9ea4dde20d973073825fd2d341393b27623e7ae9fec04f0a265447"} Jul 09 14:04:51.861559 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:51.861541 2573 scope.go:117] "RemoveContainer" containerID="39809398ad9ea4dde20d973073825fd2d341393b27623e7ae9fec04f0a265447" Jul 09 14:04:51.862655 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:51.862624 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:04:51.862763 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:51.862710 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:04:51.862827 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:51.862774 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:04:51.973406 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:51.973370 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:04:52.784873 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:52.784832 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:52.868479 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:52.868442 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" event={"ID":"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994","Type":"ContainerStarted","Data":"00d34ffed2769b917d3366b16ce70b7d1fc3aa851ede51083f77250f63bb25c5"} Jul 09 14:04:53.599542 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:53.599496 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:53.600727 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:53.600683 2573 patch_prober.go:28] interesting pod/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 09 14:04:53.600882 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:53.600770 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" podUID="1c7d3cc8-84b2-40c0-a5ac-14fee9eff994" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 09 14:04:54.599933 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:54.599881 2573 patch_prober.go:28] interesting pod/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 09 14:04:54.600486 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:54.599973 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" podUID="1c7d3cc8-84b2-40c0-a5ac-14fee9eff994" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 09 14:04:55.604188 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:55.604146 2573 patch_prober.go:28] interesting pod/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 09 14:04:55.604807 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:55.604767 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" podUID="1c7d3cc8-84b2-40c0-a5ac-14fee9eff994" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 09 14:04:56.599861 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:56.599813 2573 patch_prober.go:28] interesting pod/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 09 14:04:56.600191 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:56.600159 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" podUID="1c7d3cc8-84b2-40c0-a5ac-14fee9eff994" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 09 14:04:57.604835 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:57.604804 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:57.605318 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:57.605133 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:04:57.605908 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:04:57.605891 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:05:17.474525 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:05:17.474478 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:05:36.881980 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:05:36.881897 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:06:07.876206 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:07.876159 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:06:09.996557 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:09.996515 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m"] Jul 09 14:06:09.996970 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:09.996732 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" podUID="1c7d3cc8-84b2-40c0-a5ac-14fee9eff994" containerName="istio-proxy" containerID="cri-o://00d34ffed2769b917d3366b16ce70b7d1fc3aa851ede51083f77250f63bb25c5" gracePeriod=30 Jul 09 14:06:12.607930 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:12.607885 2573 patch_prober.go:28] interesting pod/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 14:06:12.608368 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:12.607946 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" podUID="1c7d3cc8-84b2-40c0-a5ac-14fee9eff994" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 14:06:15.250991 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.250955 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:06:15.330054 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.329925 2573 generic.go:356] "Generic (PLEG): container finished" podID="1c7d3cc8-84b2-40c0-a5ac-14fee9eff994" containerID="00d34ffed2769b917d3366b16ce70b7d1fc3aa851ede51083f77250f63bb25c5" exitCode=0 Jul 09 14:06:15.330054 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.329981 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" Jul 09 14:06:15.330054 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.330039 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" event={"ID":"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994","Type":"ContainerDied","Data":"00d34ffed2769b917d3366b16ce70b7d1fc3aa851ede51083f77250f63bb25c5"} Jul 09 14:06:15.330296 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.330096 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m" event={"ID":"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994","Type":"ContainerDied","Data":"8d5a0be32a7b6b40280f80148a5d4740a4edd7372036dc52d5318977970aa328"} Jul 09 14:06:15.330296 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.330118 2573 scope.go:117] "RemoveContainer" containerID="00d34ffed2769b917d3366b16ce70b7d1fc3aa851ede51083f77250f63bb25c5" Jul 09 14:06:15.332371 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.332352 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-credential-socket\") pod \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " Jul 09 14:06:15.332461 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.332384 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istio-podinfo\") pod \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " Jul 09 14:06:15.332461 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.332410 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-workload-certs\") pod \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " Jul 09 14:06:15.332461 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.332445 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istio-data\") pod \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " Jul 09 14:06:15.332588 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.332476 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istiod-ca-cert\") pod \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " Jul 09 14:06:15.332647 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.332607 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1c7d3cc8-84b2-40c0-a5ac-14fee9eff994" (UID: "1c7d3cc8-84b2-40c0-a5ac-14fee9eff994"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:06:15.332647 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.332624 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istio-envoy\") pod \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " Jul 09 14:06:15.332793 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.332766 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1c7d3cc8-84b2-40c0-a5ac-14fee9eff994" (UID: "1c7d3cc8-84b2-40c0-a5ac-14fee9eff994"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:06:15.332793 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.332785 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vs442\" (UniqueName: \"kubernetes.io/projected/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-kube-api-access-vs442\") pod \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " Jul 09 14:06:15.332925 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.332822 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-workload-socket\") pod \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " Jul 09 14:06:15.332925 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.332870 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istio-token\") pod \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\" (UID: \"1c7d3cc8-84b2-40c0-a5ac-14fee9eff994\") " Jul 09 14:06:15.332925 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.332894 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1c7d3cc8-84b2-40c0-a5ac-14fee9eff994" (UID: "1c7d3cc8-84b2-40c0-a5ac-14fee9eff994"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:06:15.333199 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.333176 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1c7d3cc8-84b2-40c0-a5ac-14fee9eff994" (UID: "1c7d3cc8-84b2-40c0-a5ac-14fee9eff994"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:06:15.333330 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.333310 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-workload-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:06:15.333400 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.333338 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-credential-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:06:15.333400 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.333353 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-workload-certs\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:06:15.333400 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.333367 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istiod-ca-cert\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:06:15.333559 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.333539 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istio-data" (OuterVolumeSpecName: "istio-data") pod "1c7d3cc8-84b2-40c0-a5ac-14fee9eff994" (UID: "1c7d3cc8-84b2-40c0-a5ac-14fee9eff994"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:06:15.335065 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.335036 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-kube-api-access-vs442" (OuterVolumeSpecName: "kube-api-access-vs442") pod "1c7d3cc8-84b2-40c0-a5ac-14fee9eff994" (UID: "1c7d3cc8-84b2-40c0-a5ac-14fee9eff994"). InnerVolumeSpecName "kube-api-access-vs442". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:06:15.335336 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.335308 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1c7d3cc8-84b2-40c0-a5ac-14fee9eff994" (UID: "1c7d3cc8-84b2-40c0-a5ac-14fee9eff994"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 14:06:15.335336 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.335322 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1c7d3cc8-84b2-40c0-a5ac-14fee9eff994" (UID: "1c7d3cc8-84b2-40c0-a5ac-14fee9eff994"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:06:15.335513 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.335472 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istio-token" (OuterVolumeSpecName: "istio-token") pod "1c7d3cc8-84b2-40c0-a5ac-14fee9eff994" (UID: "1c7d3cc8-84b2-40c0-a5ac-14fee9eff994"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:06:15.357460 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.357431 2573 scope.go:117] "RemoveContainer" containerID="39809398ad9ea4dde20d973073825fd2d341393b27623e7ae9fec04f0a265447" Jul 09 14:06:15.366346 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.366325 2573 scope.go:117] "RemoveContainer" containerID="00d34ffed2769b917d3366b16ce70b7d1fc3aa851ede51083f77250f63bb25c5" Jul 09 14:06:15.366701 ip-10-0-142-87 kubenswrapper[2573]: E0709 14:06:15.366666 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00d34ffed2769b917d3366b16ce70b7d1fc3aa851ede51083f77250f63bb25c5\": container with ID starting with 00d34ffed2769b917d3366b16ce70b7d1fc3aa851ede51083f77250f63bb25c5 not found: ID does not exist" containerID="00d34ffed2769b917d3366b16ce70b7d1fc3aa851ede51083f77250f63bb25c5" Jul 09 14:06:15.366761 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.366702 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00d34ffed2769b917d3366b16ce70b7d1fc3aa851ede51083f77250f63bb25c5"} err="failed to get container status \"00d34ffed2769b917d3366b16ce70b7d1fc3aa851ede51083f77250f63bb25c5\": rpc error: code = NotFound desc = could not find container \"00d34ffed2769b917d3366b16ce70b7d1fc3aa851ede51083f77250f63bb25c5\": container with ID starting with 00d34ffed2769b917d3366b16ce70b7d1fc3aa851ede51083f77250f63bb25c5 not found: ID does not exist" Jul 09 14:06:15.366761 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.366725 2573 scope.go:117] "RemoveContainer" containerID="39809398ad9ea4dde20d973073825fd2d341393b27623e7ae9fec04f0a265447" Jul 09 14:06:15.366996 ip-10-0-142-87 kubenswrapper[2573]: E0709 14:06:15.366978 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39809398ad9ea4dde20d973073825fd2d341393b27623e7ae9fec04f0a265447\": container with ID starting with 39809398ad9ea4dde20d973073825fd2d341393b27623e7ae9fec04f0a265447 not found: ID does not exist" containerID="39809398ad9ea4dde20d973073825fd2d341393b27623e7ae9fec04f0a265447" Jul 09 14:06:15.367085 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.367001 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39809398ad9ea4dde20d973073825fd2d341393b27623e7ae9fec04f0a265447"} err="failed to get container status \"39809398ad9ea4dde20d973073825fd2d341393b27623e7ae9fec04f0a265447\": rpc error: code = NotFound desc = could not find container \"39809398ad9ea4dde20d973073825fd2d341393b27623e7ae9fec04f0a265447\": container with ID starting with 39809398ad9ea4dde20d973073825fd2d341393b27623e7ae9fec04f0a265447 not found: ID does not exist" Jul 09 14:06:15.434141 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.434094 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vs442\" (UniqueName: \"kubernetes.io/projected/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-kube-api-access-vs442\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:06:15.434141 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.434131 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istio-token\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:06:15.434141 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.434145 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istio-podinfo\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:06:15.434433 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.434155 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istio-data\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:06:15.434433 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.434167 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994-istio-envoy\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:06:15.654950 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.654905 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m"] Jul 09 14:06:15.659525 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:15.659493 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-reuse-d43b1d-openshift-default-567f98dcbc-9jb6m"] Jul 09 14:06:16.945930 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:16.945893 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c7d3cc8-84b2-40c0-a5ac-14fee9eff994" path="/var/lib/kubelet/pods/1c7d3cc8-84b2-40c0-a5ac-14fee9eff994/volumes" Jul 09 14:06:24.372315 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:24.372273 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:06:55.990788 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:55.990750 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd"] Jul 09 14:06:55.991279 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:55.991035 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" podUID="f42268e9-719a-4a80-bf65-134f5087e17f" containerName="istio-proxy" containerID="cri-o://16f519362d6d79367e3bfb1d9a50294dd9e9fa4aa9ba04d7c0fa0201a816c4f3" gracePeriod=30 Jul 09 14:06:59.659687 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:59.659647 2573 patch_prober.go:28] interesting pod/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 14:06:59.660092 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:06:59.659705 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" podUID="f42268e9-719a-4a80-bf65-134f5087e17f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 14:07:00.174888 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:00.174847 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:07:01.263647 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.263571 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:07:01.397139 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.397100 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-istio-data\") pod \"f42268e9-719a-4a80-bf65-134f5087e17f\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " Jul 09 14:07:01.397139 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.397139 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-istio-envoy\") pod \"f42268e9-719a-4a80-bf65-134f5087e17f\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " Jul 09 14:07:01.397408 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.397174 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f42268e9-719a-4a80-bf65-134f5087e17f-istiod-ca-cert\") pod \"f42268e9-719a-4a80-bf65-134f5087e17f\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " Jul 09 14:07:01.397408 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.397194 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-workload-socket\") pod \"f42268e9-719a-4a80-bf65-134f5087e17f\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " Jul 09 14:07:01.397408 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.397336 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f42268e9-719a-4a80-bf65-134f5087e17f-istio-token\") pod \"f42268e9-719a-4a80-bf65-134f5087e17f\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " Jul 09 14:07:01.397583 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.397453 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9q68w\" (UniqueName: \"kubernetes.io/projected/f42268e9-719a-4a80-bf65-134f5087e17f-kube-api-access-9q68w\") pod \"f42268e9-719a-4a80-bf65-134f5087e17f\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " Jul 09 14:07:01.397583 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.397503 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-workload-certs\") pod \"f42268e9-719a-4a80-bf65-134f5087e17f\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " Jul 09 14:07:01.397583 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.397514 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f42268e9-719a-4a80-bf65-134f5087e17f" (UID: "f42268e9-719a-4a80-bf65-134f5087e17f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:07:01.397742 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.397597 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-credential-socket\") pod \"f42268e9-719a-4a80-bf65-134f5087e17f\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " Jul 09 14:07:01.397742 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.397622 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f42268e9-719a-4a80-bf65-134f5087e17f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f42268e9-719a-4a80-bf65-134f5087e17f" (UID: "f42268e9-719a-4a80-bf65-134f5087e17f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:07:01.397742 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.397640 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f42268e9-719a-4a80-bf65-134f5087e17f-istio-podinfo\") pod \"f42268e9-719a-4a80-bf65-134f5087e17f\" (UID: \"f42268e9-719a-4a80-bf65-134f5087e17f\") " Jul 09 14:07:01.397899 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.397790 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f42268e9-719a-4a80-bf65-134f5087e17f" (UID: "f42268e9-719a-4a80-bf65-134f5087e17f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:07:01.397957 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.397906 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f42268e9-719a-4a80-bf65-134f5087e17f" (UID: "f42268e9-719a-4a80-bf65-134f5087e17f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:07:01.398076 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.398054 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-istio-data" (OuterVolumeSpecName: "istio-data") pod "f42268e9-719a-4a80-bf65-134f5087e17f" (UID: "f42268e9-719a-4a80-bf65-134f5087e17f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:07:01.398188 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.398077 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-credential-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:01.398188 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.398103 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f42268e9-719a-4a80-bf65-134f5087e17f-istiod-ca-cert\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:01.398188 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.398123 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-workload-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:01.398188 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.398136 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-workload-certs\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:01.399548 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.399522 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f42268e9-719a-4a80-bf65-134f5087e17f" (UID: "f42268e9-719a-4a80-bf65-134f5087e17f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:07:01.399750 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.399735 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f42268e9-719a-4a80-bf65-134f5087e17f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f42268e9-719a-4a80-bf65-134f5087e17f" (UID: "f42268e9-719a-4a80-bf65-134f5087e17f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 14:07:01.399978 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.399956 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f42268e9-719a-4a80-bf65-134f5087e17f-istio-token" (OuterVolumeSpecName: "istio-token") pod "f42268e9-719a-4a80-bf65-134f5087e17f" (UID: "f42268e9-719a-4a80-bf65-134f5087e17f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:07:01.400059 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.399974 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f42268e9-719a-4a80-bf65-134f5087e17f-kube-api-access-9q68w" (OuterVolumeSpecName: "kube-api-access-9q68w") pod "f42268e9-719a-4a80-bf65-134f5087e17f" (UID: "f42268e9-719a-4a80-bf65-134f5087e17f"). InnerVolumeSpecName "kube-api-access-9q68w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:07:01.499401 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.499362 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f42268e9-719a-4a80-bf65-134f5087e17f-istio-token\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:01.499401 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.499396 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9q68w\" (UniqueName: \"kubernetes.io/projected/f42268e9-719a-4a80-bf65-134f5087e17f-kube-api-access-9q68w\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:01.499401 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.499406 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f42268e9-719a-4a80-bf65-134f5087e17f-istio-podinfo\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:01.499401 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.499415 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-istio-data\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:01.499684 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.499423 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f42268e9-719a-4a80-bf65-134f5087e17f-istio-envoy\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:01.572784 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.572694 2573 generic.go:356] "Generic (PLEG): container finished" podID="f42268e9-719a-4a80-bf65-134f5087e17f" containerID="16f519362d6d79367e3bfb1d9a50294dd9e9fa4aa9ba04d7c0fa0201a816c4f3" exitCode=0 Jul 09 14:07:01.572784 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.572748 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" Jul 09 14:07:01.573058 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.572791 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" event={"ID":"f42268e9-719a-4a80-bf65-134f5087e17f","Type":"ContainerDied","Data":"16f519362d6d79367e3bfb1d9a50294dd9e9fa4aa9ba04d7c0fa0201a816c4f3"} Jul 09 14:07:01.573058 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.572857 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd" event={"ID":"f42268e9-719a-4a80-bf65-134f5087e17f","Type":"ContainerDied","Data":"bc43139f8cfe0862cbd09ae351d2983266e9f1456302651265bd98ec4ea3fd9f"} Jul 09 14:07:01.573058 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.572889 2573 scope.go:117] "RemoveContainer" containerID="16f519362d6d79367e3bfb1d9a50294dd9e9fa4aa9ba04d7c0fa0201a816c4f3" Jul 09 14:07:01.586627 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.586591 2573 scope.go:117] "RemoveContainer" containerID="8b912bb855db51f89b563db4f398e18ed8ef08dcd08006d588fc2a27bd3890e8" Jul 09 14:07:01.595642 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.595619 2573 scope.go:117] "RemoveContainer" containerID="16f519362d6d79367e3bfb1d9a50294dd9e9fa4aa9ba04d7c0fa0201a816c4f3" Jul 09 14:07:01.595935 ip-10-0-142-87 kubenswrapper[2573]: E0709 14:07:01.595914 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16f519362d6d79367e3bfb1d9a50294dd9e9fa4aa9ba04d7c0fa0201a816c4f3\": container with ID starting with 16f519362d6d79367e3bfb1d9a50294dd9e9fa4aa9ba04d7c0fa0201a816c4f3 not found: ID does not exist" containerID="16f519362d6d79367e3bfb1d9a50294dd9e9fa4aa9ba04d7c0fa0201a816c4f3" Jul 09 14:07:01.595980 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.595946 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16f519362d6d79367e3bfb1d9a50294dd9e9fa4aa9ba04d7c0fa0201a816c4f3"} err="failed to get container status \"16f519362d6d79367e3bfb1d9a50294dd9e9fa4aa9ba04d7c0fa0201a816c4f3\": rpc error: code = NotFound desc = could not find container \"16f519362d6d79367e3bfb1d9a50294dd9e9fa4aa9ba04d7c0fa0201a816c4f3\": container with ID starting with 16f519362d6d79367e3bfb1d9a50294dd9e9fa4aa9ba04d7c0fa0201a816c4f3 not found: ID does not exist" Jul 09 14:07:01.595980 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.595964 2573 scope.go:117] "RemoveContainer" containerID="8b912bb855db51f89b563db4f398e18ed8ef08dcd08006d588fc2a27bd3890e8" Jul 09 14:07:01.596337 ip-10-0-142-87 kubenswrapper[2573]: E0709 14:07:01.596321 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b912bb855db51f89b563db4f398e18ed8ef08dcd08006d588fc2a27bd3890e8\": container with ID starting with 8b912bb855db51f89b563db4f398e18ed8ef08dcd08006d588fc2a27bd3890e8 not found: ID does not exist" containerID="8b912bb855db51f89b563db4f398e18ed8ef08dcd08006d588fc2a27bd3890e8" Jul 09 14:07:01.596390 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.596342 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b912bb855db51f89b563db4f398e18ed8ef08dcd08006d588fc2a27bd3890e8"} err="failed to get container status \"8b912bb855db51f89b563db4f398e18ed8ef08dcd08006d588fc2a27bd3890e8\": rpc error: code = NotFound desc = could not find container \"8b912bb855db51f89b563db4f398e18ed8ef08dcd08006d588fc2a27bd3890e8\": container with ID starting with 8b912bb855db51f89b563db4f398e18ed8ef08dcd08006d588fc2a27bd3890e8 not found: ID does not exist" Jul 09 14:07:01.603475 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.603442 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd"] Jul 09 14:07:01.607869 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:01.607840 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-440311-openshift-default-ccdbb99d7-zkpsd"] Jul 09 14:07:02.945999 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:02.945966 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f42268e9-719a-4a80-bf65-134f5087e17f" path="/var/lib/kubelet/pods/f42268e9-719a-4a80-bf65-134f5087e17f/volumes" Jul 09 14:07:03.613591 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:03.613550 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n"] Jul 09 14:07:03.613838 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:03.613812 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" podUID="9663bf3a-fae2-4cc9-aa82-166afd101bda" containerName="istio-proxy" containerID="cri-o://8cfcb5e2192dc88f47491442bb3618f59e9f771d997055571d1ce04c0456eea6" gracePeriod=30 Jul 09 14:07:04.335361 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.335276 2573 patch_prober.go:28] interesting pod/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 14:07:04.335361 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.335349 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" podUID="9663bf3a-fae2-4cc9-aa82-166afd101bda" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 14:07:04.492770 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.492736 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9"] Jul 09 14:07:04.493151 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.493133 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c7d3cc8-84b2-40c0-a5ac-14fee9eff994" containerName="istio-proxy" Jul 09 14:07:04.493151 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.493149 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c7d3cc8-84b2-40c0-a5ac-14fee9eff994" containerName="istio-proxy" Jul 09 14:07:04.493330 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.493163 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec10af80-f607-4ce3-a666-85a31996898e" containerName="istio-proxy" Jul 09 14:07:04.493330 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.493169 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec10af80-f607-4ce3-a666-85a31996898e" containerName="istio-proxy" Jul 09 14:07:04.493330 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.493179 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f42268e9-719a-4a80-bf65-134f5087e17f" containerName="istio-proxy" Jul 09 14:07:04.493330 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.493186 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f42268e9-719a-4a80-bf65-134f5087e17f" containerName="istio-proxy" Jul 09 14:07:04.493330 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.493202 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f42268e9-719a-4a80-bf65-134f5087e17f" containerName="istio-proxy" Jul 09 14:07:04.493330 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.493208 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f42268e9-719a-4a80-bf65-134f5087e17f" containerName="istio-proxy" Jul 09 14:07:04.493330 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.493221 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec10af80-f607-4ce3-a666-85a31996898e" containerName="istio-proxy" Jul 09 14:07:04.493330 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.493227 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec10af80-f607-4ce3-a666-85a31996898e" containerName="istio-proxy" Jul 09 14:07:04.493330 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.493279 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec10af80-f607-4ce3-a666-85a31996898e" containerName="istio-proxy" Jul 09 14:07:04.493330 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.493287 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f42268e9-719a-4a80-bf65-134f5087e17f" containerName="istio-proxy" Jul 09 14:07:04.493330 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.493294 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f42268e9-719a-4a80-bf65-134f5087e17f" containerName="istio-proxy" Jul 09 14:07:04.493330 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.493300 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec10af80-f607-4ce3-a666-85a31996898e" containerName="istio-proxy" Jul 09 14:07:04.493330 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.493310 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c7d3cc8-84b2-40c0-a5ac-14fee9eff994" containerName="istio-proxy" Jul 09 14:07:04.493330 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.493315 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c7d3cc8-84b2-40c0-a5ac-14fee9eff994" containerName="istio-proxy" Jul 09 14:07:04.497943 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.497922 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.500685 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.500664 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-5c1b98-openshift-default-dockercfg-bmcss\"" Jul 09 14:07:04.512370 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.512337 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9"] Jul 09 14:07:04.630619 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.630523 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-istio-envoy\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.630619 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.630568 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6403777d-537c-4163-931d-4673955261af-istiod-ca-cert\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.630884 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.630639 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-workload-certs\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.630884 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.630681 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-credential-socket\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.630884 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.630782 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6403777d-537c-4163-931d-4673955261af-istio-token\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.630884 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.630819 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-istio-data\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.630884 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.630847 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxmh8\" (UniqueName: \"kubernetes.io/projected/6403777d-537c-4163-931d-4673955261af-kube-api-access-qxmh8\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.630884 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.630880 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-workload-socket\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.631142 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.630963 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6403777d-537c-4163-931d-4673955261af-istio-podinfo\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.731669 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.731630 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-credential-socket\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.731875 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.731693 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6403777d-537c-4163-931d-4673955261af-istio-token\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.731875 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.731713 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-istio-data\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.731875 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.731740 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxmh8\" (UniqueName: \"kubernetes.io/projected/6403777d-537c-4163-931d-4673955261af-kube-api-access-qxmh8\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.731875 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.731762 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-workload-socket\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.731875 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.731834 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6403777d-537c-4163-931d-4673955261af-istio-podinfo\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.732164 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.731867 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-istio-envoy\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.732164 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.731906 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6403777d-537c-4163-931d-4673955261af-istiod-ca-cert\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.732164 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.731966 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-workload-certs\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.732278 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.732197 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-credential-socket\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.732378 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.732356 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-istio-data\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.732440 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.732379 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-workload-socket\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.732532 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.732499 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-workload-certs\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.732730 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.732711 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6403777d-537c-4163-931d-4673955261af-istiod-ca-cert\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.734366 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.734336 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-istio-envoy\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.734451 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.734438 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6403777d-537c-4163-931d-4673955261af-istio-podinfo\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.742228 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.742195 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6403777d-537c-4163-931d-4673955261af-istio-token\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.742371 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.742348 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxmh8\" (UniqueName: \"kubernetes.io/projected/6403777d-537c-4163-931d-4673955261af-kube-api-access-qxmh8\") pod \"e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.808803 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.808772 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:04.955740 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.955715 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9"] Jul 09 14:07:04.959190 ip-10-0-142-87 kubenswrapper[2573]: W0709 14:07:04.959159 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6403777d_537c_4163_931d_4673955261af.slice/crio-5da0bd1c1a1e3a6f2a75de9aa2aadebffd2701a44ad3468a78c1794c62dd4565 WatchSource:0}: Error finding container 5da0bd1c1a1e3a6f2a75de9aa2aadebffd2701a44ad3468a78c1794c62dd4565: Status 404 returned error can't find the container with id 5da0bd1c1a1e3a6f2a75de9aa2aadebffd2701a44ad3468a78c1794c62dd4565 Jul 09 14:07:04.961849 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.961815 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:07:04.961958 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.961881 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:07:04.961958 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:04.961912 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:07:05.599349 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:05.599310 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" event={"ID":"6403777d-537c-4163-931d-4673955261af","Type":"ContainerStarted","Data":"e6a1cb92b1388351af643f97e9881c10620c8f0bb1f7ecebc6d31cf2064938a2"} Jul 09 14:07:05.599810 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:05.599404 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" event={"ID":"6403777d-537c-4163-931d-4673955261af","Type":"ContainerStarted","Data":"5da0bd1c1a1e3a6f2a75de9aa2aadebffd2701a44ad3468a78c1794c62dd4565"} Jul 09 14:07:05.621070 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:05.620982 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" podStartSLOduration=1.620961824 podStartE2EDuration="1.620961824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:07:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:07:05.620871564 +0000 UTC m=+2243.126173831" watchObservedRunningTime="2026-07-09 14:07:05.620961824 +0000 UTC m=+2243.126264093" Jul 09 14:07:05.809203 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:05.809165 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:06.814488 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:06.814456 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:07.610271 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:07.610235 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:07.611388 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:07.611368 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:08.877661 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:08.877585 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:07:08.977904 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:08.977865 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-istio-data\") pod \"9663bf3a-fae2-4cc9-aa82-166afd101bda\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " Jul 09 14:07:08.978145 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:08.977912 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9663bf3a-fae2-4cc9-aa82-166afd101bda-istio-podinfo\") pod \"9663bf3a-fae2-4cc9-aa82-166afd101bda\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " Jul 09 14:07:08.978145 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:08.977953 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9663bf3a-fae2-4cc9-aa82-166afd101bda-istiod-ca-cert\") pod \"9663bf3a-fae2-4cc9-aa82-166afd101bda\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " Jul 09 14:07:08.978145 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:08.977980 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-credential-socket\") pod \"9663bf3a-fae2-4cc9-aa82-166afd101bda\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " Jul 09 14:07:08.978145 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:08.978009 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-workload-certs\") pod \"9663bf3a-fae2-4cc9-aa82-166afd101bda\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " Jul 09 14:07:08.978145 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:08.978074 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-istio-envoy\") pod \"9663bf3a-fae2-4cc9-aa82-166afd101bda\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " Jul 09 14:07:08.978442 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:08.978159 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9663bf3a-fae2-4cc9-aa82-166afd101bda-istio-token\") pod \"9663bf3a-fae2-4cc9-aa82-166afd101bda\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " Jul 09 14:07:08.978442 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:08.978194 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrn6l\" (UniqueName: \"kubernetes.io/projected/9663bf3a-fae2-4cc9-aa82-166afd101bda-kube-api-access-rrn6l\") pod \"9663bf3a-fae2-4cc9-aa82-166afd101bda\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " Jul 09 14:07:08.978442 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:08.978238 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-workload-socket\") pod \"9663bf3a-fae2-4cc9-aa82-166afd101bda\" (UID: \"9663bf3a-fae2-4cc9-aa82-166afd101bda\") " Jul 09 14:07:08.978442 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:08.978332 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9663bf3a-fae2-4cc9-aa82-166afd101bda" (UID: "9663bf3a-fae2-4cc9-aa82-166afd101bda"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:07:08.978660 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:08.978630 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-credential-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:08.978858 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:08.978826 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-istio-data" (OuterVolumeSpecName: "istio-data") pod "9663bf3a-fae2-4cc9-aa82-166afd101bda" (UID: "9663bf3a-fae2-4cc9-aa82-166afd101bda"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:07:08.979003 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:08.978903 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9663bf3a-fae2-4cc9-aa82-166afd101bda-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9663bf3a-fae2-4cc9-aa82-166afd101bda" (UID: "9663bf3a-fae2-4cc9-aa82-166afd101bda"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:07:08.979201 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:08.979174 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9663bf3a-fae2-4cc9-aa82-166afd101bda" (UID: "9663bf3a-fae2-4cc9-aa82-166afd101bda"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:07:08.979302 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:08.979203 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9663bf3a-fae2-4cc9-aa82-166afd101bda" (UID: "9663bf3a-fae2-4cc9-aa82-166afd101bda"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:07:08.980491 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:08.980468 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9663bf3a-fae2-4cc9-aa82-166afd101bda-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9663bf3a-fae2-4cc9-aa82-166afd101bda" (UID: "9663bf3a-fae2-4cc9-aa82-166afd101bda"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 14:07:08.980491 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:08.980473 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9663bf3a-fae2-4cc9-aa82-166afd101bda" (UID: "9663bf3a-fae2-4cc9-aa82-166afd101bda"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:07:08.981109 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:08.981088 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9663bf3a-fae2-4cc9-aa82-166afd101bda-istio-token" (OuterVolumeSpecName: "istio-token") pod "9663bf3a-fae2-4cc9-aa82-166afd101bda" (UID: "9663bf3a-fae2-4cc9-aa82-166afd101bda"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:07:08.981175 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:08.981153 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9663bf3a-fae2-4cc9-aa82-166afd101bda-kube-api-access-rrn6l" (OuterVolumeSpecName: "kube-api-access-rrn6l") pod "9663bf3a-fae2-4cc9-aa82-166afd101bda" (UID: "9663bf3a-fae2-4cc9-aa82-166afd101bda"). InnerVolumeSpecName "kube-api-access-rrn6l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:07:09.080242 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:09.080184 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9663bf3a-fae2-4cc9-aa82-166afd101bda-istio-token\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:09.080242 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:09.080239 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrn6l\" (UniqueName: \"kubernetes.io/projected/9663bf3a-fae2-4cc9-aa82-166afd101bda-kube-api-access-rrn6l\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:09.080242 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:09.080254 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-workload-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:09.080492 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:09.080266 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-istio-data\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:09.080492 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:09.080279 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9663bf3a-fae2-4cc9-aa82-166afd101bda-istio-podinfo\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:09.080492 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:09.080291 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9663bf3a-fae2-4cc9-aa82-166afd101bda-istiod-ca-cert\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:09.080492 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:09.080303 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-workload-certs\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:09.080492 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:09.080314 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9663bf3a-fae2-4cc9-aa82-166afd101bda-istio-envoy\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:09.621764 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:09.621731 2573 generic.go:356] "Generic (PLEG): container finished" podID="9663bf3a-fae2-4cc9-aa82-166afd101bda" containerID="8cfcb5e2192dc88f47491442bb3618f59e9f771d997055571d1ce04c0456eea6" exitCode=0 Jul 09 14:07:09.621980 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:09.621788 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" Jul 09 14:07:09.621980 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:09.621816 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" event={"ID":"9663bf3a-fae2-4cc9-aa82-166afd101bda","Type":"ContainerDied","Data":"8cfcb5e2192dc88f47491442bb3618f59e9f771d997055571d1ce04c0456eea6"} Jul 09 14:07:09.621980 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:09.621878 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n" event={"ID":"9663bf3a-fae2-4cc9-aa82-166afd101bda","Type":"ContainerDied","Data":"ec401b6b32cfaada91cdfe60cf999b5c0cf30adcc6a200e0c1204791b9c4cbfd"} Jul 09 14:07:09.621980 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:09.621908 2573 scope.go:117] "RemoveContainer" containerID="8cfcb5e2192dc88f47491442bb3618f59e9f771d997055571d1ce04c0456eea6" Jul 09 14:07:09.631999 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:09.631981 2573 scope.go:117] "RemoveContainer" containerID="deadf063bde5c31f8ef06fbc4c6bce71f67a2ac5c16ba06bd646a80cee7384ab" Jul 09 14:07:09.640272 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:09.640250 2573 scope.go:117] "RemoveContainer" containerID="8cfcb5e2192dc88f47491442bb3618f59e9f771d997055571d1ce04c0456eea6" Jul 09 14:07:09.640585 ip-10-0-142-87 kubenswrapper[2573]: E0709 14:07:09.640560 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cfcb5e2192dc88f47491442bb3618f59e9f771d997055571d1ce04c0456eea6\": container with ID starting with 8cfcb5e2192dc88f47491442bb3618f59e9f771d997055571d1ce04c0456eea6 not found: ID does not exist" containerID="8cfcb5e2192dc88f47491442bb3618f59e9f771d997055571d1ce04c0456eea6" Jul 09 14:07:09.640697 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:09.640592 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cfcb5e2192dc88f47491442bb3618f59e9f771d997055571d1ce04c0456eea6"} err="failed to get container status \"8cfcb5e2192dc88f47491442bb3618f59e9f771d997055571d1ce04c0456eea6\": rpc error: code = NotFound desc = could not find container \"8cfcb5e2192dc88f47491442bb3618f59e9f771d997055571d1ce04c0456eea6\": container with ID starting with 8cfcb5e2192dc88f47491442bb3618f59e9f771d997055571d1ce04c0456eea6 not found: ID does not exist" Jul 09 14:07:09.640697 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:09.640612 2573 scope.go:117] "RemoveContainer" containerID="deadf063bde5c31f8ef06fbc4c6bce71f67a2ac5c16ba06bd646a80cee7384ab" Jul 09 14:07:09.640832 ip-10-0-142-87 kubenswrapper[2573]: E0709 14:07:09.640815 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"deadf063bde5c31f8ef06fbc4c6bce71f67a2ac5c16ba06bd646a80cee7384ab\": container with ID starting with deadf063bde5c31f8ef06fbc4c6bce71f67a2ac5c16ba06bd646a80cee7384ab not found: ID does not exist" containerID="deadf063bde5c31f8ef06fbc4c6bce71f67a2ac5c16ba06bd646a80cee7384ab" Jul 09 14:07:09.640871 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:09.640836 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"deadf063bde5c31f8ef06fbc4c6bce71f67a2ac5c16ba06bd646a80cee7384ab"} err="failed to get container status \"deadf063bde5c31f8ef06fbc4c6bce71f67a2ac5c16ba06bd646a80cee7384ab\": rpc error: code = NotFound desc = could not find container \"deadf063bde5c31f8ef06fbc4c6bce71f67a2ac5c16ba06bd646a80cee7384ab\": container with ID starting with deadf063bde5c31f8ef06fbc4c6bce71f67a2ac5c16ba06bd646a80cee7384ab not found: ID does not exist" Jul 09 14:07:09.645776 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:09.645748 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n"] Jul 09 14:07:09.652726 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:09.652696 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-505c59-openshift-default-7b74df9f57-j7t9n"] Jul 09 14:07:10.946322 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:10.946280 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9663bf3a-fae2-4cc9-aa82-166afd101bda" path="/var/lib/kubelet/pods/9663bf3a-fae2-4cc9-aa82-166afd101bda/volumes" Jul 09 14:07:12.821308 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:12.821276 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf"] Jul 09 14:07:12.821894 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:12.821866 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9663bf3a-fae2-4cc9-aa82-166afd101bda" containerName="istio-proxy" Jul 09 14:07:12.821894 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:12.821895 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9663bf3a-fae2-4cc9-aa82-166afd101bda" containerName="istio-proxy" Jul 09 14:07:12.822091 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:12.821917 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9663bf3a-fae2-4cc9-aa82-166afd101bda" containerName="istio-proxy" Jul 09 14:07:12.822091 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:12.821927 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9663bf3a-fae2-4cc9-aa82-166afd101bda" containerName="istio-proxy" Jul 09 14:07:12.822091 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:12.821940 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c7d3cc8-84b2-40c0-a5ac-14fee9eff994" containerName="istio-proxy" Jul 09 14:07:12.822091 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:12.821950 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c7d3cc8-84b2-40c0-a5ac-14fee9eff994" containerName="istio-proxy" Jul 09 14:07:12.822091 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:12.822078 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="9663bf3a-fae2-4cc9-aa82-166afd101bda" containerName="istio-proxy" Jul 09 14:07:12.822091 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:12.822094 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="9663bf3a-fae2-4cc9-aa82-166afd101bda" containerName="istio-proxy" Jul 09 14:07:12.826342 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:12.826311 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:12.829090 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:12.829064 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-606620-openshift-default-dockercfg-f98r6\"" Jul 09 14:07:12.837695 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:12.837664 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf"] Jul 09 14:07:12.917550 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:12.917510 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-workload-certs\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:12.917736 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:12.917559 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-credential-socket\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:12.917736 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:12.917630 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-workload-socket\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:12.917736 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:12.917705 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-istio-data\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:12.917874 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:12.917758 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c82ef50-08e7-48d9-8525-fe372b143da4-istio-token\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:12.917874 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:12.917782 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-istio-envoy\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:12.917874 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:12.917807 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c82ef50-08e7-48d9-8525-fe372b143da4-istio-podinfo\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:12.918055 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:12.917903 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q658s\" (UniqueName: \"kubernetes.io/projected/2c82ef50-08e7-48d9-8525-fe372b143da4-kube-api-access-q658s\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:12.918055 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:12.917955 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c82ef50-08e7-48d9-8525-fe372b143da4-istiod-ca-cert\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:13.019254 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.019203 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-istio-data\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:13.019467 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.019279 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c82ef50-08e7-48d9-8525-fe372b143da4-istio-token\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:13.019467 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.019309 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-istio-envoy\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:13.019467 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.019330 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c82ef50-08e7-48d9-8525-fe372b143da4-istio-podinfo\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:13.019467 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.019378 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q658s\" (UniqueName: \"kubernetes.io/projected/2c82ef50-08e7-48d9-8525-fe372b143da4-kube-api-access-q658s\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:13.019467 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.019413 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c82ef50-08e7-48d9-8525-fe372b143da4-istiod-ca-cert\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:13.019741 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.019484 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-workload-certs\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:13.019741 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.019657 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-istio-data\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:13.019937 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.019911 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-credential-socket\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:13.020117 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.019950 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-workload-certs\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:13.020117 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.019996 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-workload-socket\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:13.020256 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.020235 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-credential-socket\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:13.020333 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.020265 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c82ef50-08e7-48d9-8525-fe372b143da4-istiod-ca-cert\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:13.020333 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.020324 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-workload-socket\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:13.022312 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.022257 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-istio-envoy\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:13.023248 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.023217 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c82ef50-08e7-48d9-8525-fe372b143da4-istio-podinfo\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:13.028766 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.028732 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c82ef50-08e7-48d9-8525-fe372b143da4-istio-token\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:13.031066 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.030980 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q658s\" (UniqueName: \"kubernetes.io/projected/2c82ef50-08e7-48d9-8525-fe372b143da4-kube-api-access-q658s\") pod \"e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:13.138136 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.138104 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:13.296262 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.296236 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf"] Jul 09 14:07:13.298566 ip-10-0-142-87 kubenswrapper[2573]: W0709 14:07:13.298521 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c82ef50_08e7_48d9_8525_fe372b143da4.slice/crio-19bad511eb203e1e7ceb0858be1e5891ca08e0804872cc4d266be7bd86b35e37 WatchSource:0}: Error finding container 19bad511eb203e1e7ceb0858be1e5891ca08e0804872cc4d266be7bd86b35e37: Status 404 returned error can't find the container with id 19bad511eb203e1e7ceb0858be1e5891ca08e0804872cc4d266be7bd86b35e37 Jul 09 14:07:13.300850 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.300808 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:07:13.300944 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.300882 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:07:13.300944 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.300915 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:07:13.644120 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.644083 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" event={"ID":"2c82ef50-08e7-48d9-8525-fe372b143da4","Type":"ContainerStarted","Data":"80f82ec1d313a0412d3f84040121bbb9bbd0b016969ab48431f37e100b1181da"} Jul 09 14:07:13.644327 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.644137 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" event={"ID":"2c82ef50-08e7-48d9-8525-fe372b143da4","Type":"ContainerStarted","Data":"19bad511eb203e1e7ceb0858be1e5891ca08e0804872cc4d266be7bd86b35e37"} Jul 09 14:07:13.667737 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:13.667682 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" podStartSLOduration=1.667665785 podStartE2EDuration="1.667665785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:07:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:07:13.66520809 +0000 UTC m=+2251.170510420" watchObservedRunningTime="2026-07-09 14:07:13.667665785 +0000 UTC m=+2251.172968120" Jul 09 14:07:14.138658 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:14.138629 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:14.143565 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:14.143538 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:14.649271 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:14.649245 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:14.650276 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:14.650251 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:15.369483 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:15.369448 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:07:23.652218 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:23.652168 2573 patch_prober.go:28] interesting pod/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 14:07:23.652701 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:23.652266 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" podUID="6403777d-537c-4163-931d-4673955261af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 14:07:25.713823 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:25.713788 2573 generic.go:356] "Generic (PLEG): container finished" podID="6403777d-537c-4163-931d-4673955261af" containerID="e6a1cb92b1388351af643f97e9881c10620c8f0bb1f7ecebc6d31cf2064938a2" exitCode=0 Jul 09 14:07:25.714291 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:25.713860 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" event={"ID":"6403777d-537c-4163-931d-4673955261af","Type":"ContainerDied","Data":"e6a1cb92b1388351af643f97e9881c10620c8f0bb1f7ecebc6d31cf2064938a2"} Jul 09 14:07:25.714336 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:25.714313 2573 scope.go:117] "RemoveContainer" containerID="e6a1cb92b1388351af643f97e9881c10620c8f0bb1f7ecebc6d31cf2064938a2" Jul 09 14:07:25.715345 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:25.715312 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:07:25.715465 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:25.715398 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:07:25.715465 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:25.715444 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:07:26.726207 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:26.726163 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" event={"ID":"6403777d-537c-4163-931d-4673955261af","Type":"ContainerStarted","Data":"dca327691aec0655b14579888b8be77c43d318aab14d1b0a9f5c2ab7422935e1"} Jul 09 14:07:26.809634 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:26.809597 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:26.809968 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:26.809753 2573 patch_prober.go:28] interesting pod/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 09 14:07:26.810214 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:26.810168 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" podUID="6403777d-537c-4163-931d-4673955261af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 09 14:07:27.810156 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:27.809657 2573 patch_prober.go:28] interesting pod/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 09 14:07:27.810156 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:27.809727 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" podUID="6403777d-537c-4163-931d-4673955261af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 09 14:07:28.810186 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:28.810144 2573 patch_prober.go:28] interesting pod/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 09 14:07:28.810780 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:28.810744 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" podUID="6403777d-537c-4163-931d-4673955261af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 09 14:07:29.810002 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:29.809957 2573 patch_prober.go:28] interesting pod/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 09 14:07:29.810213 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:29.810054 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" podUID="6403777d-537c-4163-931d-4673955261af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 09 14:07:30.072206 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:30.072106 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:07:30.690241 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:30.690193 2573 patch_prober.go:28] interesting pod/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 14:07:30.690441 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:30.690257 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" podUID="2c82ef50-08e7-48d9-8525-fe372b143da4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 14:07:30.813368 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:30.813335 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:30.813784 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:30.813629 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:30.814655 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:30.814627 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:32.763275 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:32.763188 2573 generic.go:356] "Generic (PLEG): container finished" podID="2c82ef50-08e7-48d9-8525-fe372b143da4" containerID="80f82ec1d313a0412d3f84040121bbb9bbd0b016969ab48431f37e100b1181da" exitCode=0 Jul 09 14:07:32.763696 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:32.763265 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" event={"ID":"2c82ef50-08e7-48d9-8525-fe372b143da4","Type":"ContainerDied","Data":"80f82ec1d313a0412d3f84040121bbb9bbd0b016969ab48431f37e100b1181da"} Jul 09 14:07:32.763696 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:32.763683 2573 scope.go:117] "RemoveContainer" containerID="80f82ec1d313a0412d3f84040121bbb9bbd0b016969ab48431f37e100b1181da" Jul 09 14:07:32.764705 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:32.764674 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:07:32.764812 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:32.764741 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:07:32.764812 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:32.764774 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:07:33.770674 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:33.770636 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" event={"ID":"2c82ef50-08e7-48d9-8525-fe372b143da4","Type":"ContainerStarted","Data":"f2ff88d0332494a18bb07e3c5910ea0f14078b56f794c69a405dcbda4edd3cda"} Jul 09 14:07:34.138867 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:34.138836 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:34.139126 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:34.139102 2573 patch_prober.go:28] interesting pod/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 09 14:07:34.139178 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:34.139153 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" podUID="2c82ef50-08e7-48d9-8525-fe372b143da4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 09 14:07:35.139419 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:35.139382 2573 patch_prober.go:28] interesting pod/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 09 14:07:35.139898 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:35.139443 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" podUID="2c82ef50-08e7-48d9-8525-fe372b143da4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 09 14:07:36.138984 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:36.138944 2573 patch_prober.go:28] interesting pod/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 09 14:07:36.139215 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:36.139008 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" podUID="2c82ef50-08e7-48d9-8525-fe372b143da4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 09 14:07:37.139469 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:37.139424 2573 patch_prober.go:28] interesting pod/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 09 14:07:37.139899 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:37.139504 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" podUID="2c82ef50-08e7-48d9-8525-fe372b143da4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 09 14:07:38.142293 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:38.142254 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:38.142747 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:38.142594 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:38.143328 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:38.143311 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:40.362789 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:40.362753 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9"] Jul 09 14:07:40.363241 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:40.362996 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" podUID="6403777d-537c-4163-931d-4673955261af" containerName="istio-proxy" containerID="cri-o://dca327691aec0655b14579888b8be77c43d318aab14d1b0a9f5c2ab7422935e1" gracePeriod=30 Jul 09 14:07:44.974324 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:44.974286 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:07:45.619235 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.619209 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:45.780426 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.780372 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-istio-envoy\") pod \"6403777d-537c-4163-931d-4673955261af\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " Jul 09 14:07:45.780426 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.780424 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-credential-socket\") pod \"6403777d-537c-4163-931d-4673955261af\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " Jul 09 14:07:45.780665 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.780452 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-workload-certs\") pod \"6403777d-537c-4163-931d-4673955261af\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " Jul 09 14:07:45.780665 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.780525 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-istio-data\") pod \"6403777d-537c-4163-931d-4673955261af\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " Jul 09 14:07:45.780665 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.780563 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6403777d-537c-4163-931d-4673955261af-istio-token\") pod \"6403777d-537c-4163-931d-4673955261af\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " Jul 09 14:07:45.780665 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.780589 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6403777d-537c-4163-931d-4673955261af-istio-podinfo\") pod \"6403777d-537c-4163-931d-4673955261af\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " Jul 09 14:07:45.780665 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.780618 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-workload-socket\") pod \"6403777d-537c-4163-931d-4673955261af\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " Jul 09 14:07:45.780927 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.780688 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxmh8\" (UniqueName: \"kubernetes.io/projected/6403777d-537c-4163-931d-4673955261af-kube-api-access-qxmh8\") pod \"6403777d-537c-4163-931d-4673955261af\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " Jul 09 14:07:45.780927 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.780730 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6403777d-537c-4163-931d-4673955261af-istiod-ca-cert\") pod \"6403777d-537c-4163-931d-4673955261af\" (UID: \"6403777d-537c-4163-931d-4673955261af\") " Jul 09 14:07:45.780927 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.780741 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6403777d-537c-4163-931d-4673955261af" (UID: "6403777d-537c-4163-931d-4673955261af"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:07:45.780927 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.780834 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6403777d-537c-4163-931d-4673955261af" (UID: "6403777d-537c-4163-931d-4673955261af"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:07:45.781211 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.780928 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6403777d-537c-4163-931d-4673955261af" (UID: "6403777d-537c-4163-931d-4673955261af"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:07:45.781211 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.781165 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-workload-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:45.781211 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.781186 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-credential-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:45.781211 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.781201 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-workload-certs\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:45.781423 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.781251 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6403777d-537c-4163-931d-4673955261af-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6403777d-537c-4163-931d-4673955261af" (UID: "6403777d-537c-4163-931d-4673955261af"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:07:45.781952 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.781921 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-istio-data" (OuterVolumeSpecName: "istio-data") pod "6403777d-537c-4163-931d-4673955261af" (UID: "6403777d-537c-4163-931d-4673955261af"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:07:45.782829 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.782800 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6403777d-537c-4163-931d-4673955261af-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6403777d-537c-4163-931d-4673955261af" (UID: "6403777d-537c-4163-931d-4673955261af"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 14:07:45.782934 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.782870 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6403777d-537c-4163-931d-4673955261af" (UID: "6403777d-537c-4163-931d-4673955261af"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:07:45.782934 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.782926 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6403777d-537c-4163-931d-4673955261af-kube-api-access-qxmh8" (OuterVolumeSpecName: "kube-api-access-qxmh8") pod "6403777d-537c-4163-931d-4673955261af" (UID: "6403777d-537c-4163-931d-4673955261af"). InnerVolumeSpecName "kube-api-access-qxmh8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:07:45.783044 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.782942 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6403777d-537c-4163-931d-4673955261af-istio-token" (OuterVolumeSpecName: "istio-token") pod "6403777d-537c-4163-931d-4673955261af" (UID: "6403777d-537c-4163-931d-4673955261af"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:07:45.840835 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.840800 2573 generic.go:356] "Generic (PLEG): container finished" podID="6403777d-537c-4163-931d-4673955261af" containerID="dca327691aec0655b14579888b8be77c43d318aab14d1b0a9f5c2ab7422935e1" exitCode=0 Jul 09 14:07:45.841057 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.840836 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" event={"ID":"6403777d-537c-4163-931d-4673955261af","Type":"ContainerDied","Data":"dca327691aec0655b14579888b8be77c43d318aab14d1b0a9f5c2ab7422935e1"} Jul 09 14:07:45.841057 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.840854 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" Jul 09 14:07:45.841057 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.840904 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9" event={"ID":"6403777d-537c-4163-931d-4673955261af","Type":"ContainerDied","Data":"5da0bd1c1a1e3a6f2a75de9aa2aadebffd2701a44ad3468a78c1794c62dd4565"} Jul 09 14:07:45.841057 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.840927 2573 scope.go:117] "RemoveContainer" containerID="dca327691aec0655b14579888b8be77c43d318aab14d1b0a9f5c2ab7422935e1" Jul 09 14:07:45.850491 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.850471 2573 scope.go:117] "RemoveContainer" containerID="e6a1cb92b1388351af643f97e9881c10620c8f0bb1f7ecebc6d31cf2064938a2" Jul 09 14:07:45.859370 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.859350 2573 scope.go:117] "RemoveContainer" containerID="dca327691aec0655b14579888b8be77c43d318aab14d1b0a9f5c2ab7422935e1" Jul 09 14:07:45.859679 ip-10-0-142-87 kubenswrapper[2573]: E0709 14:07:45.859655 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dca327691aec0655b14579888b8be77c43d318aab14d1b0a9f5c2ab7422935e1\": container with ID starting with dca327691aec0655b14579888b8be77c43d318aab14d1b0a9f5c2ab7422935e1 not found: ID does not exist" containerID="dca327691aec0655b14579888b8be77c43d318aab14d1b0a9f5c2ab7422935e1" Jul 09 14:07:45.859736 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.859689 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dca327691aec0655b14579888b8be77c43d318aab14d1b0a9f5c2ab7422935e1"} err="failed to get container status \"dca327691aec0655b14579888b8be77c43d318aab14d1b0a9f5c2ab7422935e1\": rpc error: code = NotFound desc = could not find container \"dca327691aec0655b14579888b8be77c43d318aab14d1b0a9f5c2ab7422935e1\": container with ID starting with dca327691aec0655b14579888b8be77c43d318aab14d1b0a9f5c2ab7422935e1 not found: ID does not exist" Jul 09 14:07:45.859736 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.859708 2573 scope.go:117] "RemoveContainer" containerID="e6a1cb92b1388351af643f97e9881c10620c8f0bb1f7ecebc6d31cf2064938a2" Jul 09 14:07:45.859965 ip-10-0-142-87 kubenswrapper[2573]: E0709 14:07:45.859944 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6a1cb92b1388351af643f97e9881c10620c8f0bb1f7ecebc6d31cf2064938a2\": container with ID starting with e6a1cb92b1388351af643f97e9881c10620c8f0bb1f7ecebc6d31cf2064938a2 not found: ID does not exist" containerID="e6a1cb92b1388351af643f97e9881c10620c8f0bb1f7ecebc6d31cf2064938a2" Jul 09 14:07:45.860039 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.859974 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6a1cb92b1388351af643f97e9881c10620c8f0bb1f7ecebc6d31cf2064938a2"} err="failed to get container status \"e6a1cb92b1388351af643f97e9881c10620c8f0bb1f7ecebc6d31cf2064938a2\": rpc error: code = NotFound desc = could not find container \"e6a1cb92b1388351af643f97e9881c10620c8f0bb1f7ecebc6d31cf2064938a2\": container with ID starting with e6a1cb92b1388351af643f97e9881c10620c8f0bb1f7ecebc6d31cf2064938a2 not found: ID does not exist" Jul 09 14:07:45.865135 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.865104 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9"] Jul 09 14:07:45.870133 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.870089 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-5c1b98-openshift-default-5fb674859-mqxw9"] Jul 09 14:07:45.882572 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.882524 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-istio-envoy\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:45.882572 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.882565 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6403777d-537c-4163-931d-4673955261af-istio-data\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:45.882572 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.882573 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6403777d-537c-4163-931d-4673955261af-istio-token\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:45.882572 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.882581 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6403777d-537c-4163-931d-4673955261af-istio-podinfo\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:45.882833 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.882592 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxmh8\" (UniqueName: \"kubernetes.io/projected/6403777d-537c-4163-931d-4673955261af-kube-api-access-qxmh8\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:45.882833 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:45.882604 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6403777d-537c-4163-931d-4673955261af-istiod-ca-cert\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:46.945780 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:46.945737 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6403777d-537c-4163-931d-4673955261af" path="/var/lib/kubelet/pods/6403777d-537c-4163-931d-4673955261af/volumes" Jul 09 14:07:48.084394 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:48.084354 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf"] Jul 09 14:07:48.084893 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:48.084577 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" podUID="2c82ef50-08e7-48d9-8525-fe372b143da4" containerName="istio-proxy" containerID="cri-o://f2ff88d0332494a18bb07e3c5910ea0f14078b56f794c69a405dcbda4edd3cda" gracePeriod=30 Jul 09 14:07:48.949798 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:48.949754 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw"] Jul 09 14:07:48.950361 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:48.950334 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6403777d-537c-4163-931d-4673955261af" containerName="istio-proxy" Jul 09 14:07:48.950361 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:48.950362 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="6403777d-537c-4163-931d-4673955261af" containerName="istio-proxy" Jul 09 14:07:48.950570 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:48.950381 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6403777d-537c-4163-931d-4673955261af" containerName="istio-proxy" Jul 09 14:07:48.950570 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:48.950391 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="6403777d-537c-4163-931d-4673955261af" containerName="istio-proxy" Jul 09 14:07:48.950570 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:48.950514 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="6403777d-537c-4163-931d-4673955261af" containerName="istio-proxy" Jul 09 14:07:48.950570 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:48.950527 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="6403777d-537c-4163-931d-4673955261af" containerName="istio-proxy" Jul 09 14:07:48.952607 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:48.952585 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:48.955126 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:48.955104 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-06548f-openshift-default-dockercfg-xch6m\"" Jul 09 14:07:48.966395 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:48.966362 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw"] Jul 09 14:07:49.115857 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.115815 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istio-token\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.116321 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.115866 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-credential-socket\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.116321 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.115917 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istio-data\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.116321 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.115975 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istio-envoy\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.116321 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.116094 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-workload-socket\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.116321 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.116124 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dprnn\" (UniqueName: \"kubernetes.io/projected/e5951f78-28b5-49a5-96bf-e05c1f3253ee-kube-api-access-dprnn\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.116321 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.116149 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istiod-ca-cert\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.116321 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.116204 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istio-podinfo\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.116321 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.116232 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-workload-certs\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.217829 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.217473 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-workload-socket\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.217829 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.217536 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dprnn\" (UniqueName: \"kubernetes.io/projected/e5951f78-28b5-49a5-96bf-e05c1f3253ee-kube-api-access-dprnn\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.217829 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.217724 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istiod-ca-cert\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.217829 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.217781 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istio-podinfo\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.217829 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.217815 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-workload-certs\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.218273 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.217903 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istio-token\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.218273 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.217908 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-workload-socket\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.218273 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.217944 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-credential-socket\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.218273 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.217990 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istio-data\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.218273 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.218095 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istio-envoy\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.218273 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.218198 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-workload-certs\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.218584 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.218442 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istio-data\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.218642 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.218612 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istiod-ca-cert\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.218700 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.218662 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-credential-socket\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.221658 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.221630 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istio-podinfo\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.222142 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.222098 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istio-envoy\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.227599 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.227571 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istio-token\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.228321 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.228300 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dprnn\" (UniqueName: \"kubernetes.io/projected/e5951f78-28b5-49a5-96bf-e05c1f3253ee-kube-api-access-dprnn\") pod \"e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.264137 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.264084 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:49.410282 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.410250 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw"] Jul 09 14:07:49.412349 ip-10-0-142-87 kubenswrapper[2573]: W0709 14:07:49.412323 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5951f78_28b5_49a5_96bf_e05c1f3253ee.slice/crio-a505c381bf8fc68eb2a84d57d9f78db31257cbdbca4b4e56becaccca97cba535 WatchSource:0}: Error finding container a505c381bf8fc68eb2a84d57d9f78db31257cbdbca4b4e56becaccca97cba535: Status 404 returned error can't find the container with id a505c381bf8fc68eb2a84d57d9f78db31257cbdbca4b4e56becaccca97cba535 Jul 09 14:07:49.415242 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.415199 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:07:49.415340 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.415285 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:07:49.415340 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.415323 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:07:49.863626 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.863583 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" event={"ID":"e5951f78-28b5-49a5-96bf-e05c1f3253ee","Type":"ContainerStarted","Data":"6f1ba68e0aaf544686d183bd48c701ccef66d539f86ccc81db08ed20dff89374"} Jul 09 14:07:49.863785 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.863687 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" event={"ID":"e5951f78-28b5-49a5-96bf-e05c1f3253ee","Type":"ContainerStarted","Data":"a505c381bf8fc68eb2a84d57d9f78db31257cbdbca4b4e56becaccca97cba535"} Jul 09 14:07:49.887418 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:49.887358 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" podStartSLOduration=1.8873416650000001 podStartE2EDuration="1.887341665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:07:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:07:49.884633631 +0000 UTC m=+2287.389935900" watchObservedRunningTime="2026-07-09 14:07:49.887341665 +0000 UTC m=+2287.392643934" Jul 09 14:07:50.264418 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:50.264376 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:50.269555 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:50.269526 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:50.868435 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:50.868401 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:50.869400 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:50.869380 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:07:53.143703 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.143667 2573 patch_prober.go:28] interesting pod/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 09 14:07:53.144084 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.143724 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" podUID="2c82ef50-08e7-48d9-8525-fe372b143da4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 09 14:07:53.345881 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.345817 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:53.360554 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.360517 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-istio-envoy\") pod \"2c82ef50-08e7-48d9-8525-fe372b143da4\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " Jul 09 14:07:53.360554 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.360549 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-istio-data\") pod \"2c82ef50-08e7-48d9-8525-fe372b143da4\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " Jul 09 14:07:53.360801 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.360572 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c82ef50-08e7-48d9-8525-fe372b143da4-istiod-ca-cert\") pod \"2c82ef50-08e7-48d9-8525-fe372b143da4\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " Jul 09 14:07:53.360801 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.360616 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c82ef50-08e7-48d9-8525-fe372b143da4-istio-podinfo\") pod \"2c82ef50-08e7-48d9-8525-fe372b143da4\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " Jul 09 14:07:53.360801 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.360696 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-workload-socket\") pod \"2c82ef50-08e7-48d9-8525-fe372b143da4\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " Jul 09 14:07:53.360801 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.360742 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-credential-socket\") pod \"2c82ef50-08e7-48d9-8525-fe372b143da4\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " Jul 09 14:07:53.360801 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.360780 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-workload-certs\") pod \"2c82ef50-08e7-48d9-8525-fe372b143da4\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " Jul 09 14:07:53.361067 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.360810 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c82ef50-08e7-48d9-8525-fe372b143da4-istio-token\") pod \"2c82ef50-08e7-48d9-8525-fe372b143da4\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " Jul 09 14:07:53.361067 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.360873 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q658s\" (UniqueName: \"kubernetes.io/projected/2c82ef50-08e7-48d9-8525-fe372b143da4-kube-api-access-q658s\") pod \"2c82ef50-08e7-48d9-8525-fe372b143da4\" (UID: \"2c82ef50-08e7-48d9-8525-fe372b143da4\") " Jul 09 14:07:53.361178 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.361073 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c82ef50-08e7-48d9-8525-fe372b143da4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2c82ef50-08e7-48d9-8525-fe372b143da4" (UID: "2c82ef50-08e7-48d9-8525-fe372b143da4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:07:53.361368 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.361327 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2c82ef50-08e7-48d9-8525-fe372b143da4" (UID: "2c82ef50-08e7-48d9-8525-fe372b143da4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:07:53.361464 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.361412 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c82ef50-08e7-48d9-8525-fe372b143da4-istiod-ca-cert\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:53.361464 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.361437 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-istio-data" (OuterVolumeSpecName: "istio-data") pod "2c82ef50-08e7-48d9-8525-fe372b143da4" (UID: "2c82ef50-08e7-48d9-8525-fe372b143da4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:07:53.361669 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.361637 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2c82ef50-08e7-48d9-8525-fe372b143da4" (UID: "2c82ef50-08e7-48d9-8525-fe372b143da4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:07:53.361669 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.361649 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2c82ef50-08e7-48d9-8525-fe372b143da4" (UID: "2c82ef50-08e7-48d9-8525-fe372b143da4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:07:53.363014 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.362988 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2c82ef50-08e7-48d9-8525-fe372b143da4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2c82ef50-08e7-48d9-8525-fe372b143da4" (UID: "2c82ef50-08e7-48d9-8525-fe372b143da4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 14:07:53.363201 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.363175 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c82ef50-08e7-48d9-8525-fe372b143da4-kube-api-access-q658s" (OuterVolumeSpecName: "kube-api-access-q658s") pod "2c82ef50-08e7-48d9-8525-fe372b143da4" (UID: "2c82ef50-08e7-48d9-8525-fe372b143da4"). InnerVolumeSpecName "kube-api-access-q658s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:07:53.363270 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.363249 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2c82ef50-08e7-48d9-8525-fe372b143da4" (UID: "2c82ef50-08e7-48d9-8525-fe372b143da4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:07:53.363528 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.363509 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c82ef50-08e7-48d9-8525-fe372b143da4-istio-token" (OuterVolumeSpecName: "istio-token") pod "2c82ef50-08e7-48d9-8525-fe372b143da4" (UID: "2c82ef50-08e7-48d9-8525-fe372b143da4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:07:53.462842 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.462810 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-workload-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:53.462842 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.462837 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-credential-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:53.462842 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.462846 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-workload-certs\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:53.463109 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.462854 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c82ef50-08e7-48d9-8525-fe372b143da4-istio-token\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:53.463109 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.462863 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q658s\" (UniqueName: \"kubernetes.io/projected/2c82ef50-08e7-48d9-8525-fe372b143da4-kube-api-access-q658s\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:53.463109 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.462871 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-istio-envoy\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:53.463109 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.462884 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c82ef50-08e7-48d9-8525-fe372b143da4-istio-data\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:53.463109 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.462895 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c82ef50-08e7-48d9-8525-fe372b143da4-istio-podinfo\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:07:53.885535 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.885498 2573 generic.go:356] "Generic (PLEG): container finished" podID="2c82ef50-08e7-48d9-8525-fe372b143da4" containerID="f2ff88d0332494a18bb07e3c5910ea0f14078b56f794c69a405dcbda4edd3cda" exitCode=0 Jul 09 14:07:53.885716 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.885550 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" Jul 09 14:07:53.885716 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.885578 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" event={"ID":"2c82ef50-08e7-48d9-8525-fe372b143da4","Type":"ContainerDied","Data":"f2ff88d0332494a18bb07e3c5910ea0f14078b56f794c69a405dcbda4edd3cda"} Jul 09 14:07:53.885716 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.885630 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf" event={"ID":"2c82ef50-08e7-48d9-8525-fe372b143da4","Type":"ContainerDied","Data":"19bad511eb203e1e7ceb0858be1e5891ca08e0804872cc4d266be7bd86b35e37"} Jul 09 14:07:53.885716 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.885651 2573 scope.go:117] "RemoveContainer" containerID="f2ff88d0332494a18bb07e3c5910ea0f14078b56f794c69a405dcbda4edd3cda" Jul 09 14:07:53.895994 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.895974 2573 scope.go:117] "RemoveContainer" containerID="80f82ec1d313a0412d3f84040121bbb9bbd0b016969ab48431f37e100b1181da" Jul 09 14:07:53.904724 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.904703 2573 scope.go:117] "RemoveContainer" containerID="f2ff88d0332494a18bb07e3c5910ea0f14078b56f794c69a405dcbda4edd3cda" Jul 09 14:07:53.905044 ip-10-0-142-87 kubenswrapper[2573]: E0709 14:07:53.905008 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2ff88d0332494a18bb07e3c5910ea0f14078b56f794c69a405dcbda4edd3cda\": container with ID starting with f2ff88d0332494a18bb07e3c5910ea0f14078b56f794c69a405dcbda4edd3cda not found: ID does not exist" containerID="f2ff88d0332494a18bb07e3c5910ea0f14078b56f794c69a405dcbda4edd3cda" Jul 09 14:07:53.905106 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.905057 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2ff88d0332494a18bb07e3c5910ea0f14078b56f794c69a405dcbda4edd3cda"} err="failed to get container status \"f2ff88d0332494a18bb07e3c5910ea0f14078b56f794c69a405dcbda4edd3cda\": rpc error: code = NotFound desc = could not find container \"f2ff88d0332494a18bb07e3c5910ea0f14078b56f794c69a405dcbda4edd3cda\": container with ID starting with f2ff88d0332494a18bb07e3c5910ea0f14078b56f794c69a405dcbda4edd3cda not found: ID does not exist" Jul 09 14:07:53.905106 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.905077 2573 scope.go:117] "RemoveContainer" containerID="80f82ec1d313a0412d3f84040121bbb9bbd0b016969ab48431f37e100b1181da" Jul 09 14:07:53.905308 ip-10-0-142-87 kubenswrapper[2573]: E0709 14:07:53.905292 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80f82ec1d313a0412d3f84040121bbb9bbd0b016969ab48431f37e100b1181da\": container with ID starting with 80f82ec1d313a0412d3f84040121bbb9bbd0b016969ab48431f37e100b1181da not found: ID does not exist" containerID="80f82ec1d313a0412d3f84040121bbb9bbd0b016969ab48431f37e100b1181da" Jul 09 14:07:53.905350 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.905314 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80f82ec1d313a0412d3f84040121bbb9bbd0b016969ab48431f37e100b1181da"} err="failed to get container status \"80f82ec1d313a0412d3f84040121bbb9bbd0b016969ab48431f37e100b1181da\": rpc error: code = NotFound desc = could not find container \"80f82ec1d313a0412d3f84040121bbb9bbd0b016969ab48431f37e100b1181da\": container with ID starting with 80f82ec1d313a0412d3f84040121bbb9bbd0b016969ab48431f37e100b1181da not found: ID does not exist" Jul 09 14:07:53.914319 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.914287 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf"] Jul 09 14:07:53.919269 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:53.919235 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-606620-openshift-default-7778dc5d6f-c7dkf"] Jul 09 14:07:54.946068 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:54.946010 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c82ef50-08e7-48d9-8525-fe372b143da4" path="/var/lib/kubelet/pods/2c82ef50-08e7-48d9-8525-fe372b143da4/volumes" Jul 09 14:07:56.988294 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:56.988256 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx"] Jul 09 14:07:56.988847 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:56.988829 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c82ef50-08e7-48d9-8525-fe372b143da4" containerName="istio-proxy" Jul 09 14:07:56.988847 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:56.988849 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c82ef50-08e7-48d9-8525-fe372b143da4" containerName="istio-proxy" Jul 09 14:07:56.988972 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:56.988858 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c82ef50-08e7-48d9-8525-fe372b143da4" containerName="istio-proxy" Jul 09 14:07:56.988972 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:56.988864 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c82ef50-08e7-48d9-8525-fe372b143da4" containerName="istio-proxy" Jul 09 14:07:56.988972 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:56.988937 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c82ef50-08e7-48d9-8525-fe372b143da4" containerName="istio-proxy" Jul 09 14:07:56.988972 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:56.988947 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c82ef50-08e7-48d9-8525-fe372b143da4" containerName="istio-proxy" Jul 09 14:07:56.992556 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:56.992537 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:56.995431 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:56.995406 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-6ee5b0-openshift-default-dockercfg-rzgjg\"" Jul 09 14:07:57.005197 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.005169 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx"] Jul 09 14:07:57.104879 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.104824 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-workload-socket\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.105146 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.104896 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e029d52-1b42-48aa-98b2-6e40dbe40199-istio-podinfo\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.105146 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.104963 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfmv4\" (UniqueName: \"kubernetes.io/projected/2e029d52-1b42-48aa-98b2-6e40dbe40199-kube-api-access-pfmv4\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.105146 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.105034 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e029d52-1b42-48aa-98b2-6e40dbe40199-istio-token\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.105146 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.105065 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-istio-data\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.105146 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.105098 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e029d52-1b42-48aa-98b2-6e40dbe40199-istiod-ca-cert\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.105425 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.105151 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-workload-certs\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.105425 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.105181 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-istio-envoy\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.105425 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.105377 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-credential-socket\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.206244 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.206208 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-workload-certs\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.206244 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.206244 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-istio-envoy\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.206476 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.206291 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-credential-socket\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.206476 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.206327 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-workload-socket\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.206476 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.206354 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e029d52-1b42-48aa-98b2-6e40dbe40199-istio-podinfo\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.206633 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.206494 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfmv4\" (UniqueName: \"kubernetes.io/projected/2e029d52-1b42-48aa-98b2-6e40dbe40199-kube-api-access-pfmv4\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.206633 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.206573 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e029d52-1b42-48aa-98b2-6e40dbe40199-istio-token\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.206633 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.206602 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-istio-data\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.206778 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.206638 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e029d52-1b42-48aa-98b2-6e40dbe40199-istiod-ca-cert\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.206778 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.206658 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-workload-certs\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.206917 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.206896 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-workload-socket\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.207082 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.207062 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-istio-data\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.207218 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.207201 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-credential-socket\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.207370 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.207354 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e029d52-1b42-48aa-98b2-6e40dbe40199-istiod-ca-cert\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.208696 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.208673 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-istio-envoy\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.208918 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.208895 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e029d52-1b42-48aa-98b2-6e40dbe40199-istio-podinfo\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.216704 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.216672 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e029d52-1b42-48aa-98b2-6e40dbe40199-istio-token\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.216857 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.216799 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfmv4\" (UniqueName: \"kubernetes.io/projected/2e029d52-1b42-48aa-98b2-6e40dbe40199-kube-api-access-pfmv4\") pod \"e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.304319 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.304228 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:57.454070 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.454037 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx"] Jul 09 14:07:57.456857 ip-10-0-142-87 kubenswrapper[2573]: W0709 14:07:57.456829 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e029d52_1b42_48aa_98b2_6e40dbe40199.slice/crio-d5b6d59119f0b2e47b2301e58b88f557b183ea9a6d6393f63ff45a14cad4f9eb WatchSource:0}: Error finding container d5b6d59119f0b2e47b2301e58b88f557b183ea9a6d6393f63ff45a14cad4f9eb: Status 404 returned error can't find the container with id d5b6d59119f0b2e47b2301e58b88f557b183ea9a6d6393f63ff45a14cad4f9eb Jul 09 14:07:57.459114 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.459075 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:07:57.459227 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.459162 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:07:57.459227 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.459207 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:07:57.914973 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.914929 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" event={"ID":"2e029d52-1b42-48aa-98b2-6e40dbe40199","Type":"ContainerStarted","Data":"b0958c8a3708d33717e666302b31b3044d01c6fae99d57712a8817e3aaf24453"} Jul 09 14:07:57.915210 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.914991 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" event={"ID":"2e029d52-1b42-48aa-98b2-6e40dbe40199","Type":"ContainerStarted","Data":"d5b6d59119f0b2e47b2301e58b88f557b183ea9a6d6393f63ff45a14cad4f9eb"} Jul 09 14:07:57.939075 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:57.939002 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" podStartSLOduration=1.938983347 podStartE2EDuration="1.938983347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:07:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:07:57.936585623 +0000 UTC m=+2295.441887908" watchObservedRunningTime="2026-07-09 14:07:57.938983347 +0000 UTC m=+2295.444285615" Jul 09 14:07:58.305060 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:58.304937 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:58.309831 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:58.309804 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:58.920075 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:58.920044 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:58.921119 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:58.921100 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:07:59.772613 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:07:59.772585 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:08:06.910224 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:06.910182 2573 patch_prober.go:28] interesting pod/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 14:08:06.910613 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:06.910243 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" podUID="e5951f78-28b5-49a5-96bf-e05c1f3253ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 14:08:07.973170 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:07.973081 2573 generic.go:356] "Generic (PLEG): container finished" podID="e5951f78-28b5-49a5-96bf-e05c1f3253ee" containerID="6f1ba68e0aaf544686d183bd48c701ccef66d539f86ccc81db08ed20dff89374" exitCode=0 Jul 09 14:08:07.973170 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:07.973152 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" event={"ID":"e5951f78-28b5-49a5-96bf-e05c1f3253ee","Type":"ContainerDied","Data":"6f1ba68e0aaf544686d183bd48c701ccef66d539f86ccc81db08ed20dff89374"} Jul 09 14:08:07.973594 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:07.973582 2573 scope.go:117] "RemoveContainer" containerID="6f1ba68e0aaf544686d183bd48c701ccef66d539f86ccc81db08ed20dff89374" Jul 09 14:08:07.974675 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:07.974643 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:08:07.974780 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:07.974762 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:08:07.974817 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:07.974807 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:08:08.980477 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:08.980440 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" event={"ID":"e5951f78-28b5-49a5-96bf-e05c1f3253ee","Type":"ContainerStarted","Data":"8851a1b1e92bd1a27f8b503c56dede4b127743b78e3db9ae08468e218fe795b7"} Jul 09 14:08:09.264684 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:09.264577 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:08:09.264858 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:09.264835 2573 patch_prober.go:28] interesting pod/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 09 14:08:09.264933 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:09.264906 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" podUID="e5951f78-28b5-49a5-96bf-e05c1f3253ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 09 14:08:10.265226 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:10.265184 2573 patch_prober.go:28] interesting pod/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 09 14:08:10.265636 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:10.265239 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" podUID="e5951f78-28b5-49a5-96bf-e05c1f3253ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 09 14:08:11.264812 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:11.264768 2573 patch_prober.go:28] interesting pod/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 09 14:08:11.265012 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:11.264825 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" podUID="e5951f78-28b5-49a5-96bf-e05c1f3253ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 09 14:08:11.470488 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:11.470444 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:08:12.265377 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:12.265340 2573 patch_prober.go:28] interesting pod/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 09 14:08:12.265583 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:12.265394 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" podUID="e5951f78-28b5-49a5-96bf-e05c1f3253ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 09 14:08:13.269560 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:13.269517 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:08:13.270653 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:13.270181 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:08:13.270885 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:13.270861 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:08:14.961226 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:14.961191 2573 patch_prober.go:28] interesting pod/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 14:08:14.962930 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:14.961260 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" podUID="2e029d52-1b42-48aa-98b2-6e40dbe40199" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 14:08:16.026717 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:16.026678 2573 generic.go:356] "Generic (PLEG): container finished" podID="2e029d52-1b42-48aa-98b2-6e40dbe40199" containerID="b0958c8a3708d33717e666302b31b3044d01c6fae99d57712a8817e3aaf24453" exitCode=0 Jul 09 14:08:16.027132 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:16.026751 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" event={"ID":"2e029d52-1b42-48aa-98b2-6e40dbe40199","Type":"ContainerDied","Data":"b0958c8a3708d33717e666302b31b3044d01c6fae99d57712a8817e3aaf24453"} Jul 09 14:08:16.027201 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:16.027186 2573 scope.go:117] "RemoveContainer" containerID="b0958c8a3708d33717e666302b31b3044d01c6fae99d57712a8817e3aaf24453" Jul 09 14:08:16.028227 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:16.028200 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:08:16.028291 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:16.028270 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:08:16.028327 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:16.028305 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 14:08:17.035451 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:17.035412 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" event={"ID":"2e029d52-1b42-48aa-98b2-6e40dbe40199","Type":"ContainerStarted","Data":"a2b786129b3a6e593ff0e6f4ff7c67f353e3cef3e5ba049a4e2bc1567307e489"} Jul 09 14:08:17.305330 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:17.305231 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:08:17.305713 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:17.305611 2573 patch_prober.go:28] interesting pod/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 09 14:08:17.305898 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:17.305875 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" podUID="2e029d52-1b42-48aa-98b2-6e40dbe40199" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 09 14:08:18.306073 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:18.305546 2573 patch_prober.go:28] interesting pod/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 09 14:08:18.306073 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:18.305631 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" podUID="2e029d52-1b42-48aa-98b2-6e40dbe40199" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 09 14:08:19.305393 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:19.305343 2573 patch_prober.go:28] interesting pod/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 09 14:08:19.305616 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:19.305436 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" podUID="2e029d52-1b42-48aa-98b2-6e40dbe40199" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 09 14:08:20.305764 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:20.305309 2573 patch_prober.go:28] interesting pod/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 09 14:08:20.305764 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:20.305377 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" podUID="2e029d52-1b42-48aa-98b2-6e40dbe40199" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 09 14:08:21.308726 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:21.308694 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:08:21.309161 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:21.308953 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:08:21.309595 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:21.309579 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:08:23.478325 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:23.478284 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:08:35.000979 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:35.000887 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:08:53.315832 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:08:53.315785 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:09:11.978660 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:09:11.978619 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:09:26.971909 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:09:26.971871 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:09:43.068200 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:09:43.068168 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmms8_3b512d09-0288-4fea-864e-fdc734afb1bd/ovn-acl-logging/0.log" Jul 09 14:09:43.077248 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:09:43.077225 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmms8_3b512d09-0288-4fea-864e-fdc734afb1bd/ovn-acl-logging/0.log" Jul 09 14:09:44.576459 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:09:44.576416 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:10:02.372763 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:10:02.372721 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:10:18.868354 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:10:18.868260 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:10:37.466447 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:10:37.466408 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:10:51.669540 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:10:51.669500 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:11:10.979152 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:11:10.979115 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:11:42.177673 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:11:42.177639 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:12:16.871372 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:12:16.871331 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:12:53.375555 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:12:53.375515 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:13:11.574945 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:13:11.574907 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:13:30.769503 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:13:30.769458 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:14:21.089306 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:14:21.089268 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:14:43.100710 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:14:43.100678 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmms8_3b512d09-0288-4fea-864e-fdc734afb1bd/ovn-acl-logging/0.log" Jul 09 14:14:43.110792 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:14:43.110768 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vmms8_3b512d09-0288-4fea-864e-fdc734afb1bd/ovn-acl-logging/0.log" Jul 09 14:14:55.369331 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:14:55.369290 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:15:10.266654 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:15:10.266616 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:15:24.581709 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:15:24.581670 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:15:41.282785 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:15:41.282747 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:16:05.071431 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:05.071339 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:16:08.358167 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:08.358130 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw"] Jul 09 14:16:08.358579 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:08.358363 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" podUID="e5951f78-28b5-49a5-96bf-e05c1f3253ee" containerName="istio-proxy" containerID="cri-o://8851a1b1e92bd1a27f8b503c56dede4b127743b78e3db9ae08468e218fe795b7" gracePeriod=30 Jul 09 14:16:13.272957 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.272906 2573 patch_prober.go:28] interesting pod/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 14:16:13.273452 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.272990 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" podUID="e5951f78-28b5-49a5-96bf-e05c1f3253ee" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 14:16:13.511570 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.511537 2573 generic.go:356] "Generic (PLEG): container finished" podID="e5951f78-28b5-49a5-96bf-e05c1f3253ee" containerID="8851a1b1e92bd1a27f8b503c56dede4b127743b78e3db9ae08468e218fe795b7" exitCode=0 Jul 09 14:16:13.511776 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.511621 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" event={"ID":"e5951f78-28b5-49a5-96bf-e05c1f3253ee","Type":"ContainerDied","Data":"8851a1b1e92bd1a27f8b503c56dede4b127743b78e3db9ae08468e218fe795b7"} Jul 09 14:16:13.511776 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.511697 2573 scope.go:117] "RemoveContainer" containerID="6f1ba68e0aaf544686d183bd48c701ccef66d539f86ccc81db08ed20dff89374" Jul 09 14:16:13.622803 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.622777 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:16:13.718755 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.718724 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-workload-certs\") pod \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " Jul 09 14:16:13.718954 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.718782 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-credential-socket\") pod \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " Jul 09 14:16:13.718954 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.718801 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istiod-ca-cert\") pod \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " Jul 09 14:16:13.718954 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.718853 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dprnn\" (UniqueName: \"kubernetes.io/projected/e5951f78-28b5-49a5-96bf-e05c1f3253ee-kube-api-access-dprnn\") pod \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " Jul 09 14:16:13.718954 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.718883 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istio-envoy\") pod \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " Jul 09 14:16:13.718954 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.718914 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-workload-socket\") pod \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " Jul 09 14:16:13.719250 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.718968 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istio-podinfo\") pod \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " Jul 09 14:16:13.719250 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.719001 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istio-data\") pod \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " Jul 09 14:16:13.719250 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.719100 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e5951f78-28b5-49a5-96bf-e05c1f3253ee" (UID: "e5951f78-28b5-49a5-96bf-e05c1f3253ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:16:13.719250 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.719140 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e5951f78-28b5-49a5-96bf-e05c1f3253ee" (UID: "e5951f78-28b5-49a5-96bf-e05c1f3253ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:16:13.719250 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.719245 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istio-token\") pod \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\" (UID: \"e5951f78-28b5-49a5-96bf-e05c1f3253ee\") " Jul 09 14:16:13.719523 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.719320 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e5951f78-28b5-49a5-96bf-e05c1f3253ee" (UID: "e5951f78-28b5-49a5-96bf-e05c1f3253ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:16:13.719523 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.719314 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e5951f78-28b5-49a5-96bf-e05c1f3253ee" (UID: "e5951f78-28b5-49a5-96bf-e05c1f3253ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:16:13.719671 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.719645 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-workload-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:16:13.719752 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.719673 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-workload-certs\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:16:13.719752 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.719689 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-credential-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:16:13.719752 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.719702 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istiod-ca-cert\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:16:13.720143 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.720108 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istio-data" (OuterVolumeSpecName: "istio-data") pod "e5951f78-28b5-49a5-96bf-e05c1f3253ee" (UID: "e5951f78-28b5-49a5-96bf-e05c1f3253ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:16:13.721756 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.721713 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e5951f78-28b5-49a5-96bf-e05c1f3253ee" (UID: "e5951f78-28b5-49a5-96bf-e05c1f3253ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:16:13.721903 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.721759 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e5951f78-28b5-49a5-96bf-e05c1f3253ee" (UID: "e5951f78-28b5-49a5-96bf-e05c1f3253ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 14:16:13.721903 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.721821 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5951f78-28b5-49a5-96bf-e05c1f3253ee-kube-api-access-dprnn" (OuterVolumeSpecName: "kube-api-access-dprnn") pod "e5951f78-28b5-49a5-96bf-e05c1f3253ee" (UID: "e5951f78-28b5-49a5-96bf-e05c1f3253ee"). InnerVolumeSpecName "kube-api-access-dprnn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:16:13.722001 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.721960 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istio-token" (OuterVolumeSpecName: "istio-token") pod "e5951f78-28b5-49a5-96bf-e05c1f3253ee" (UID: "e5951f78-28b5-49a5-96bf-e05c1f3253ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:16:13.820885 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.820773 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dprnn\" (UniqueName: \"kubernetes.io/projected/e5951f78-28b5-49a5-96bf-e05c1f3253ee-kube-api-access-dprnn\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:16:13.820885 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.820817 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istio-envoy\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:16:13.820885 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.820829 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istio-podinfo\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:16:13.820885 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.820840 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istio-data\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:16:13.820885 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:13.820852 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5951f78-28b5-49a5-96bf-e05c1f3253ee-istio-token\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:16:14.518554 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:14.518527 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" Jul 09 14:16:14.518999 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:14.518571 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw" event={"ID":"e5951f78-28b5-49a5-96bf-e05c1f3253ee","Type":"ContainerDied","Data":"a505c381bf8fc68eb2a84d57d9f78db31257cbdbca4b4e56becaccca97cba535"} Jul 09 14:16:14.518999 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:14.518626 2573 scope.go:117] "RemoveContainer" containerID="8851a1b1e92bd1a27f8b503c56dede4b127743b78e3db9ae08468e218fe795b7" Jul 09 14:16:14.552705 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:14.552667 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw"] Jul 09 14:16:14.565181 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:14.565150 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-06548f-openshift-default-778d7db77f-mknnw"] Jul 09 14:16:14.946355 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:14.946318 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5951f78-28b5-49a5-96bf-e05c1f3253ee" path="/var/lib/kubelet/pods/e5951f78-28b5-49a5-96bf-e05c1f3253ee/volumes" Jul 09 14:16:16.339215 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:16.339180 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx"] Jul 09 14:16:16.339733 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:16.339408 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" podUID="2e029d52-1b42-48aa-98b2-6e40dbe40199" containerName="istio-proxy" containerID="cri-o://a2b786129b3a6e593ff0e6f4ff7c67f353e3cef3e5ba049a4e2bc1567307e489" gracePeriod=30 Jul 09 14:16:19.176541 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:19.176504 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m56nm"] Jul 09 14:16:21.311248 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.311204 2573 patch_prober.go:28] interesting pod/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 14:16:21.311633 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.311281 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" podUID="2e029d52-1b42-48aa-98b2-6e40dbe40199" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 14:16:21.558232 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.558200 2573 generic.go:356] "Generic (PLEG): container finished" podID="2e029d52-1b42-48aa-98b2-6e40dbe40199" containerID="a2b786129b3a6e593ff0e6f4ff7c67f353e3cef3e5ba049a4e2bc1567307e489" exitCode=0 Jul 09 14:16:21.558412 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.558271 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" event={"ID":"2e029d52-1b42-48aa-98b2-6e40dbe40199","Type":"ContainerDied","Data":"a2b786129b3a6e593ff0e6f4ff7c67f353e3cef3e5ba049a4e2bc1567307e489"} Jul 09 14:16:21.558412 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.558367 2573 scope.go:117] "RemoveContainer" containerID="b0958c8a3708d33717e666302b31b3044d01c6fae99d57712a8817e3aaf24453" Jul 09 14:16:21.606568 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.606541 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:16:21.692199 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.692171 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-workload-socket\") pod \"2e029d52-1b42-48aa-98b2-6e40dbe40199\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " Jul 09 14:16:21.692404 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.692210 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e029d52-1b42-48aa-98b2-6e40dbe40199-istio-token\") pod \"2e029d52-1b42-48aa-98b2-6e40dbe40199\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " Jul 09 14:16:21.692404 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.692250 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfmv4\" (UniqueName: \"kubernetes.io/projected/2e029d52-1b42-48aa-98b2-6e40dbe40199-kube-api-access-pfmv4\") pod \"2e029d52-1b42-48aa-98b2-6e40dbe40199\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " Jul 09 14:16:21.692404 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.692269 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-credential-socket\") pod \"2e029d52-1b42-48aa-98b2-6e40dbe40199\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " Jul 09 14:16:21.692404 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.692389 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e029d52-1b42-48aa-98b2-6e40dbe40199-istiod-ca-cert\") pod \"2e029d52-1b42-48aa-98b2-6e40dbe40199\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " Jul 09 14:16:21.692627 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.692420 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-istio-envoy\") pod \"2e029d52-1b42-48aa-98b2-6e40dbe40199\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " Jul 09 14:16:21.692627 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.692503 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e029d52-1b42-48aa-98b2-6e40dbe40199-istio-podinfo\") pod \"2e029d52-1b42-48aa-98b2-6e40dbe40199\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " Jul 09 14:16:21.692627 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.692537 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-istio-data\") pod \"2e029d52-1b42-48aa-98b2-6e40dbe40199\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " Jul 09 14:16:21.692627 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.692556 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-workload-certs\") pod \"2e029d52-1b42-48aa-98b2-6e40dbe40199\" (UID: \"2e029d52-1b42-48aa-98b2-6e40dbe40199\") " Jul 09 14:16:21.692627 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.692560 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2e029d52-1b42-48aa-98b2-6e40dbe40199" (UID: "2e029d52-1b42-48aa-98b2-6e40dbe40199"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:16:21.692627 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.692579 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2e029d52-1b42-48aa-98b2-6e40dbe40199" (UID: "2e029d52-1b42-48aa-98b2-6e40dbe40199"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:16:21.692916 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.692812 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e029d52-1b42-48aa-98b2-6e40dbe40199-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2e029d52-1b42-48aa-98b2-6e40dbe40199" (UID: "2e029d52-1b42-48aa-98b2-6e40dbe40199"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:16:21.692976 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.692929 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-workload-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:16:21.692976 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.692944 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-credential-socket\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:16:21.692976 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.692959 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e029d52-1b42-48aa-98b2-6e40dbe40199-istiod-ca-cert\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:16:21.693173 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.693061 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2e029d52-1b42-48aa-98b2-6e40dbe40199" (UID: "2e029d52-1b42-48aa-98b2-6e40dbe40199"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:16:21.693630 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.693604 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-istio-data" (OuterVolumeSpecName: "istio-data") pod "2e029d52-1b42-48aa-98b2-6e40dbe40199" (UID: "2e029d52-1b42-48aa-98b2-6e40dbe40199"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:16:21.694695 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.694673 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2e029d52-1b42-48aa-98b2-6e40dbe40199-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2e029d52-1b42-48aa-98b2-6e40dbe40199" (UID: "2e029d52-1b42-48aa-98b2-6e40dbe40199"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 14:16:21.694775 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.694757 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2e029d52-1b42-48aa-98b2-6e40dbe40199" (UID: "2e029d52-1b42-48aa-98b2-6e40dbe40199"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:16:21.694938 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.694916 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e029d52-1b42-48aa-98b2-6e40dbe40199-kube-api-access-pfmv4" (OuterVolumeSpecName: "kube-api-access-pfmv4") pod "2e029d52-1b42-48aa-98b2-6e40dbe40199" (UID: "2e029d52-1b42-48aa-98b2-6e40dbe40199"). InnerVolumeSpecName "kube-api-access-pfmv4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:16:21.694993 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.694934 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e029d52-1b42-48aa-98b2-6e40dbe40199-istio-token" (OuterVolumeSpecName: "istio-token") pod "2e029d52-1b42-48aa-98b2-6e40dbe40199" (UID: "2e029d52-1b42-48aa-98b2-6e40dbe40199"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:16:21.793584 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.793547 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e029d52-1b42-48aa-98b2-6e40dbe40199-istio-podinfo\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:16:21.793584 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.793577 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-istio-data\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:16:21.793584 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.793588 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-workload-certs\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:16:21.793584 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.793596 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e029d52-1b42-48aa-98b2-6e40dbe40199-istio-token\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:16:21.793865 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.793604 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfmv4\" (UniqueName: \"kubernetes.io/projected/2e029d52-1b42-48aa-98b2-6e40dbe40199-kube-api-access-pfmv4\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:16:21.793865 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:21.793613 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e029d52-1b42-48aa-98b2-6e40dbe40199-istio-envoy\") on node \"ip-10-0-142-87.ec2.internal\" DevicePath \"\"" Jul 09 14:16:22.565142 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:22.565114 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" Jul 09 14:16:22.565142 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:22.565121 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx" event={"ID":"2e029d52-1b42-48aa-98b2-6e40dbe40199","Type":"ContainerDied","Data":"d5b6d59119f0b2e47b2301e58b88f557b183ea9a6d6393f63ff45a14cad4f9eb"} Jul 09 14:16:22.565612 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:22.565173 2573 scope.go:117] "RemoveContainer" containerID="a2b786129b3a6e593ff0e6f4ff7c67f353e3cef3e5ba049a4e2bc1567307e489" Jul 09 14:16:22.589793 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:22.589761 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx"] Jul 09 14:16:22.594271 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:22.594244 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-6ee5b0-openshift-default-68d956d9bc-chthx"] Jul 09 14:16:22.897613 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:22.897588 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69bb5d6bf5-48wl2_063431c8-3f4f-47d7-bd90-1eba78202a38/copy-manifests/0.log" Jul 09 14:16:22.906546 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:22.906516 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69bb5d6bf5-48wl2_063431c8-3f4f-47d7-bd90-1eba78202a38/manager/0.log" Jul 09 14:16:22.945846 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:22.945801 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e029d52-1b42-48aa-98b2-6e40dbe40199" path="/var/lib/kubelet/pods/2e029d52-1b42-48aa-98b2-6e40dbe40199/volumes" Jul 09 14:16:26.144815 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:26.144780 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-m56nm_914a5cad-ac5f-4108-98cb-d53fd699049c/limitador/0.log" Jul 09 14:16:26.622991 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:26.622884 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4nlfp2_9dab3f11-b82c-443b-82df-a9e18f3bb094/istio-proxy/0.log" Jul 09 14:16:26.762584 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:26.762551 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-79t5l_fe184a24-50aa-4d20-8384-8aa523445d26/discovery/0.log" Jul 09 14:16:27.121223 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:27.121170 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-22g4g_623826de-9ff7-4fdb-a68d-dde99e7babb2/istio-proxy/1.log" Jul 09 14:16:38.662799 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:38.662748 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-65fnv_fb20e4df-4233-4fb6-a1f7-e58a789e3d56/global-pull-secret-syncer/0.log" Jul 09 14:16:38.903195 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:38.903163 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tnf9v_ef1861a0-ca40-4753-80c2-7085d1884d07/konnectivity-agent/0.log" Jul 09 14:16:38.990353 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:38.990264 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-87.ec2.internal_dc8175f5b3169ac412d470dea8cf3d3d/haproxy/0.log" Jul 09 14:16:44.267034 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:44.266995 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-m56nm_914a5cad-ac5f-4108-98cb-d53fd699049c/limitador/0.log" Jul 09 14:16:46.083326 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:46.083245 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f/alertmanager/0.log" Jul 09 14:16:46.132219 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:46.132183 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f/config-reloader/0.log" Jul 09 14:16:46.173993 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:46.173964 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f/kube-rbac-proxy-web/0.log" Jul 09 14:16:46.229906 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:46.229878 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f/kube-rbac-proxy/0.log" Jul 09 14:16:46.277088 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:46.277053 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f/kube-rbac-proxy-metric/0.log" Jul 09 14:16:46.324448 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:46.324416 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f/prom-label-proxy/0.log" Jul 09 14:16:46.393942 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:46.393912 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7f311ed-6ab6-4fd3-87b8-bfc6c71a0a6f/init-config-reloader/0.log" Jul 09 14:16:46.485046 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:46.484988 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-ttqkz_62b06276-f4ff-4d42-a305-339bf1c453bd/kube-state-metrics/0.log" Jul 09 14:16:46.521004 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:46.520973 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-ttqkz_62b06276-f4ff-4d42-a305-339bf1c453bd/kube-rbac-proxy-main/0.log" Jul 09 14:16:46.567091 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:46.567059 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-ttqkz_62b06276-f4ff-4d42-a305-339bf1c453bd/kube-rbac-proxy-self/0.log" Jul 09 14:16:46.610976 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:46.610888 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-77764697c6-kj8fm_43a7ad9e-a15f-46d4-8acb-791cd9a2300b/metrics-server/0.log" Jul 09 14:16:46.655898 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:46.655869 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-qblcv_2c41244e-abc4-4b1f-8c04-3d74077d0e6e/monitoring-plugin/0.log" Jul 09 14:16:46.942859 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:46.942822 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pq8wj_edec525a-47f6-4c88-92c8-4e502b6f2739/node-exporter/0.log" Jul 09 14:16:46.981992 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:46.981960 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pq8wj_edec525a-47f6-4c88-92c8-4e502b6f2739/kube-rbac-proxy/0.log" Jul 09 14:16:47.030802 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:47.030778 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pq8wj_edec525a-47f6-4c88-92c8-4e502b6f2739/init-textfile/0.log" Jul 09 14:16:47.099056 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:47.099006 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-x4jf5_151e80f3-5a05-456b-b719-41c7d7f946c6/kube-rbac-proxy-main/0.log" Jul 09 14:16:47.148922 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:47.148890 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-x4jf5_151e80f3-5a05-456b-b719-41c7d7f946c6/kube-rbac-proxy-self/0.log" Jul 09 14:16:47.196987 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:47.196901 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-x4jf5_151e80f3-5a05-456b-b719-41c7d7f946c6/openshift-state-metrics/0.log" Jul 09 14:16:47.255309 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:47.255275 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1cb1addd-9bb5-44eb-be34-0840e9f7dbfc/prometheus/0.log" Jul 09 14:16:47.307061 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:47.307010 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1cb1addd-9bb5-44eb-be34-0840e9f7dbfc/config-reloader/0.log" Jul 09 14:16:47.355167 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:47.355137 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1cb1addd-9bb5-44eb-be34-0840e9f7dbfc/thanos-sidecar/0.log" Jul 09 14:16:47.398933 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:47.398909 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1cb1addd-9bb5-44eb-be34-0840e9f7dbfc/kube-rbac-proxy-web/0.log" Jul 09 14:16:47.438006 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:47.437980 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1cb1addd-9bb5-44eb-be34-0840e9f7dbfc/kube-rbac-proxy/0.log" Jul 09 14:16:47.471345 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:47.471260 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1cb1addd-9bb5-44eb-be34-0840e9f7dbfc/kube-rbac-proxy-thanos/0.log" Jul 09 14:16:47.516487 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:47.516459 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1cb1addd-9bb5-44eb-be34-0840e9f7dbfc/init-config-reloader/0.log" Jul 09 14:16:47.586256 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:47.586228 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-lwvt4_b125e19d-1900-4c3d-945a-61de5502b779/prometheus-operator/0.log" Jul 09 14:16:47.634035 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:47.633994 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-lwvt4_b125e19d-1900-4c3d-945a-61de5502b779/kube-rbac-proxy/0.log" Jul 09 14:16:47.707144 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:47.707114 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-p5lxr_b7469638-03a3-4bd5-a128-a36e3bd5353e/prometheus-operator-admission-webhook/0.log" Jul 09 14:16:47.766227 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:47.766146 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b57485f5f-89v49_affca51d-9937-427c-9525-1a17670a9463/telemeter-client/0.log" Jul 09 14:16:47.830525 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:47.830493 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b57485f5f-89v49_affca51d-9937-427c-9525-1a17670a9463/reload/0.log" Jul 09 14:16:47.902042 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:47.901985 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b57485f5f-89v49_affca51d-9937-427c-9525-1a17670a9463/kube-rbac-proxy/0.log" Jul 09 14:16:47.954363 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:47.954333 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d5688b76d-th79t_b2d57361-d50a-47a0-9f50-d308478d019a/thanos-query/0.log" Jul 09 14:16:47.986946 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:47.986915 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d5688b76d-th79t_b2d57361-d50a-47a0-9f50-d308478d019a/kube-rbac-proxy-web/0.log" Jul 09 14:16:48.024826 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:48.024742 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d5688b76d-th79t_b2d57361-d50a-47a0-9f50-d308478d019a/kube-rbac-proxy/0.log" Jul 09 14:16:48.057445 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:48.057413 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d5688b76d-th79t_b2d57361-d50a-47a0-9f50-d308478d019a/prom-label-proxy/0.log" Jul 09 14:16:48.087618 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:48.087588 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d5688b76d-th79t_b2d57361-d50a-47a0-9f50-d308478d019a/kube-rbac-proxy-rules/0.log" Jul 09 14:16:48.117276 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:48.117242 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d5688b76d-th79t_b2d57361-d50a-47a0-9f50-d308478d019a/kube-rbac-proxy-metrics/0.log" Jul 09 14:16:48.918338 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:48.918298 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-snslw/perf-node-gather-daemonset-ckkx4"] Jul 09 14:16:48.918839 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:48.918818 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e029d52-1b42-48aa-98b2-6e40dbe40199" containerName="istio-proxy" Jul 09 14:16:48.918893 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:48.918842 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e029d52-1b42-48aa-98b2-6e40dbe40199" containerName="istio-proxy" Jul 09 14:16:48.918893 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:48.918859 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5951f78-28b5-49a5-96bf-e05c1f3253ee" containerName="istio-proxy" Jul 09 14:16:48.918893 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:48.918867 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5951f78-28b5-49a5-96bf-e05c1f3253ee" containerName="istio-proxy" Jul 09 14:16:48.919061 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:48.918890 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5951f78-28b5-49a5-96bf-e05c1f3253ee" containerName="istio-proxy" Jul 09 14:16:48.919061 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:48.918902 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5951f78-28b5-49a5-96bf-e05c1f3253ee" containerName="istio-proxy" Jul 09 14:16:48.919061 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:48.919053 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e029d52-1b42-48aa-98b2-6e40dbe40199" containerName="istio-proxy" Jul 09 14:16:48.919157 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:48.919066 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5951f78-28b5-49a5-96bf-e05c1f3253ee" containerName="istio-proxy" Jul 09 14:16:48.924088 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:48.924060 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-snslw/perf-node-gather-daemonset-ckkx4" Jul 09 14:16:48.926928 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:48.926898 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-snslw\"/\"kube-root-ca.crt\"" Jul 09 14:16:48.927215 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:48.927076 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-snslw\"/\"openshift-service-ca.crt\"" Jul 09 14:16:48.928081 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:48.928059 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-snslw\"/\"default-dockercfg-x9wk4\"" Jul 09 14:16:48.935444 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:48.935412 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-snslw/perf-node-gather-daemonset-ckkx4"] Jul 09 14:16:49.057668 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:49.057624 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdxbq\" (UniqueName: \"kubernetes.io/projected/09631b81-8325-44ce-abb8-a6740f5851bb-kube-api-access-wdxbq\") pod \"perf-node-gather-daemonset-ckkx4\" (UID: \"09631b81-8325-44ce-abb8-a6740f5851bb\") " pod="openshift-must-gather-snslw/perf-node-gather-daemonset-ckkx4" Jul 09 14:16:49.057888 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:49.057706 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/09631b81-8325-44ce-abb8-a6740f5851bb-lib-modules\") pod \"perf-node-gather-daemonset-ckkx4\" (UID: \"09631b81-8325-44ce-abb8-a6740f5851bb\") " pod="openshift-must-gather-snslw/perf-node-gather-daemonset-ckkx4" Jul 09 14:16:49.057888 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:49.057753 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/09631b81-8325-44ce-abb8-a6740f5851bb-podres\") pod \"perf-node-gather-daemonset-ckkx4\" (UID: \"09631b81-8325-44ce-abb8-a6740f5851bb\") " pod="openshift-must-gather-snslw/perf-node-gather-daemonset-ckkx4" Jul 09 14:16:49.057888 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:49.057778 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09631b81-8325-44ce-abb8-a6740f5851bb-sys\") pod \"perf-node-gather-daemonset-ckkx4\" (UID: \"09631b81-8325-44ce-abb8-a6740f5851bb\") " pod="openshift-must-gather-snslw/perf-node-gather-daemonset-ckkx4" Jul 09 14:16:49.057888 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:49.057814 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/09631b81-8325-44ce-abb8-a6740f5851bb-proc\") pod \"perf-node-gather-daemonset-ckkx4\" (UID: \"09631b81-8325-44ce-abb8-a6740f5851bb\") " pod="openshift-must-gather-snslw/perf-node-gather-daemonset-ckkx4" Jul 09 14:16:49.158926 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:49.158888 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/09631b81-8325-44ce-abb8-a6740f5851bb-proc\") pod \"perf-node-gather-daemonset-ckkx4\" (UID: \"09631b81-8325-44ce-abb8-a6740f5851bb\") " pod="openshift-must-gather-snslw/perf-node-gather-daemonset-ckkx4" Jul 09 14:16:49.159428 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:49.158968 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdxbq\" (UniqueName: \"kubernetes.io/projected/09631b81-8325-44ce-abb8-a6740f5851bb-kube-api-access-wdxbq\") pod \"perf-node-gather-daemonset-ckkx4\" (UID: \"09631b81-8325-44ce-abb8-a6740f5851bb\") " pod="openshift-must-gather-snslw/perf-node-gather-daemonset-ckkx4" Jul 09 14:16:49.159428 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:49.159008 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/09631b81-8325-44ce-abb8-a6740f5851bb-lib-modules\") pod \"perf-node-gather-daemonset-ckkx4\" (UID: \"09631b81-8325-44ce-abb8-a6740f5851bb\") " pod="openshift-must-gather-snslw/perf-node-gather-daemonset-ckkx4" Jul 09 14:16:49.159428 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:49.159051 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/09631b81-8325-44ce-abb8-a6740f5851bb-proc\") pod \"perf-node-gather-daemonset-ckkx4\" (UID: \"09631b81-8325-44ce-abb8-a6740f5851bb\") " pod="openshift-must-gather-snslw/perf-node-gather-daemonset-ckkx4" Jul 09 14:16:49.159428 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:49.159058 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/09631b81-8325-44ce-abb8-a6740f5851bb-podres\") pod \"perf-node-gather-daemonset-ckkx4\" (UID: \"09631b81-8325-44ce-abb8-a6740f5851bb\") " pod="openshift-must-gather-snslw/perf-node-gather-daemonset-ckkx4" Jul 09 14:16:49.159428 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:49.159113 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09631b81-8325-44ce-abb8-a6740f5851bb-sys\") pod \"perf-node-gather-daemonset-ckkx4\" (UID: \"09631b81-8325-44ce-abb8-a6740f5851bb\") " pod="openshift-must-gather-snslw/perf-node-gather-daemonset-ckkx4" Jul 09 14:16:49.159428 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:49.159153 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/09631b81-8325-44ce-abb8-a6740f5851bb-podres\") pod \"perf-node-gather-daemonset-ckkx4\" (UID: \"09631b81-8325-44ce-abb8-a6740f5851bb\") " pod="openshift-must-gather-snslw/perf-node-gather-daemonset-ckkx4" Jul 09 14:16:49.159428 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:49.159190 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09631b81-8325-44ce-abb8-a6740f5851bb-sys\") pod \"perf-node-gather-daemonset-ckkx4\" (UID: \"09631b81-8325-44ce-abb8-a6740f5851bb\") " pod="openshift-must-gather-snslw/perf-node-gather-daemonset-ckkx4" Jul 09 14:16:49.159428 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:49.159197 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/09631b81-8325-44ce-abb8-a6740f5851bb-lib-modules\") pod \"perf-node-gather-daemonset-ckkx4\" (UID: \"09631b81-8325-44ce-abb8-a6740f5851bb\") " pod="openshift-must-gather-snslw/perf-node-gather-daemonset-ckkx4" Jul 09 14:16:49.168533 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:49.168448 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdxbq\" (UniqueName: \"kubernetes.io/projected/09631b81-8325-44ce-abb8-a6740f5851bb-kube-api-access-wdxbq\") pod \"perf-node-gather-daemonset-ckkx4\" (UID: \"09631b81-8325-44ce-abb8-a6740f5851bb\") " pod="openshift-must-gather-snslw/perf-node-gather-daemonset-ckkx4" Jul 09 14:16:49.235793 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:49.235751 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-snslw/perf-node-gather-daemonset-ckkx4" Jul 09 14:16:49.380133 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:49.380099 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-snslw/perf-node-gather-daemonset-ckkx4"] Jul 09 14:16:49.382383 ip-10-0-142-87 kubenswrapper[2573]: W0709 14:16:49.382352 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod09631b81_8325_44ce_abb8_a6740f5851bb.slice/crio-6486bc15b801279c4700bbf105c3ba202701f7fa4b9b3587bdc8493862c2e11f WatchSource:0}: Error finding container 6486bc15b801279c4700bbf105c3ba202701f7fa4b9b3587bdc8493862c2e11f: Status 404 returned error can't find the container with id 6486bc15b801279c4700bbf105c3ba202701f7fa4b9b3587bdc8493862c2e11f Jul 09 14:16:49.708657 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:49.708557 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-snslw/perf-node-gather-daemonset-ckkx4" event={"ID":"09631b81-8325-44ce-abb8-a6740f5851bb","Type":"ContainerStarted","Data":"e423c8be9b8af61c10ebff9065d0b6cfa228f38337dcf689b8fdada53dcd8bf0"} Jul 09 14:16:49.708657 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:49.708612 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-snslw/perf-node-gather-daemonset-ckkx4" event={"ID":"09631b81-8325-44ce-abb8-a6740f5851bb","Type":"ContainerStarted","Data":"6486bc15b801279c4700bbf105c3ba202701f7fa4b9b3587bdc8493862c2e11f"} Jul 09 14:16:49.708859 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:49.708662 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-snslw/perf-node-gather-daemonset-ckkx4" Jul 09 14:16:49.728387 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:49.728328 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-snslw/perf-node-gather-daemonset-ckkx4" podStartSLOduration=1.7283044539999999 podStartE2EDuration="1.728304454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:16:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:16:49.726424882 +0000 UTC m=+2827.231727175" watchObservedRunningTime="2026-07-09 14:16:49.728304454 +0000 UTC m=+2827.233606722" Jul 09 14:16:50.519923 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:50.519891 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-78c8755b99-kbc6p_0e2abc06-b17d-40a8-a2e4-8048059b43ad/console/0.log" Jul 09 14:16:52.358454 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:52.358413 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9bhjn_07c548d3-07f4-41f8-b82c-22932794c741/dns/0.log" Jul 09 14:16:52.382727 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:52.382694 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9bhjn_07c548d3-07f4-41f8-b82c-22932794c741/kube-rbac-proxy/0.log" Jul 09 14:16:52.503704 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:52.503672 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k5fs2_c9f5a139-bec5-4097-8ca2-0618cd70bc56/dns-node-resolver/0.log" Jul 09 14:16:53.168427 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:53.168393 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fcwpf_7fc8a7a4-1f79-475a-9eed-31141361e131/node-ca/0.log" Jul 09 14:16:54.206616 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:54.206577 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4nlfp2_9dab3f11-b82c-443b-82df-a9e18f3bb094/istio-proxy/0.log" Jul 09 14:16:54.438716 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:54.438676 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-79t5l_fe184a24-50aa-4d20-8384-8aa523445d26/discovery/0.log" Jul 09 14:16:54.534761 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:54.534669 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-22g4g_623826de-9ff7-4fdb-a68d-dde99e7babb2/istio-proxy/0.log" Jul 09 14:16:54.596323 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:54.596286 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-22g4g_623826de-9ff7-4fdb-a68d-dde99e7babb2/istio-proxy/1.log" Jul 09 14:16:55.496360 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:55.496324 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qp49h_86d778c7-615a-4398-b5ff-5d4ca5cfa2e2/serve-healthcheck-canary/0.log" Jul 09 14:16:55.724600 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:55.724569 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-snslw/perf-node-gather-daemonset-ckkx4" Jul 09 14:16:56.246609 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:56.246574 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tllnp_17a28000-664b-4eba-bd87-5c67029f4358/kube-rbac-proxy/0.log" Jul 09 14:16:56.272236 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:56.272200 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tllnp_17a28000-664b-4eba-bd87-5c67029f4358/exporter/0.log" Jul 09 14:16:56.299336 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:56.299303 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tllnp_17a28000-664b-4eba-bd87-5c67029f4358/extractor/0.log" Jul 09 14:16:59.033823 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:59.033787 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4nlfp2_9dab3f11-b82c-443b-82df-a9e18f3bb094/istio-proxy/0.log" Jul 09 14:16:59.202773 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:59.202741 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69bb5d6bf5-48wl2_063431c8-3f4f-47d7-bd90-1eba78202a38/manager/0.log" Jul 09 14:16:59.233209 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:59.233173 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69bb5d6bf5-48wl2_063431c8-3f4f-47d7-bd90-1eba78202a38/copy-manifests/0.log" Jul 09 14:16:59.289175 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:59.289088 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-79t5l_fe184a24-50aa-4d20-8384-8aa523445d26/discovery/0.log" Jul 09 14:16:59.346310 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:59.346268 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-22g4g_623826de-9ff7-4fdb-a68d-dde99e7babb2/istio-proxy/0.log" Jul 09 14:16:59.405714 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:16:59.405683 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-22g4g_623826de-9ff7-4fdb-a68d-dde99e7babb2/istio-proxy/1.log" Jul 09 14:17:00.769541 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:17:00.769509 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7769c579dd-9z6lt_fb766605-dcd6-4e9e-8e4b-fef71c9fbc8f/manager/0.log" Jul 09 14:17:00.837843 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:17:00.837757 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-m2ctd_4a30c9b2-3f28-43b2-9d2d-ec88af66be0d/openshift-lws-operator/0.log" Jul 09 14:17:00.910459 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:17:00.910429 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4nlfp2_9dab3f11-b82c-443b-82df-a9e18f3bb094/istio-proxy/0.log" Jul 09 14:17:01.137763 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:17:01.137722 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-79t5l_fe184a24-50aa-4d20-8384-8aa523445d26/discovery/0.log" Jul 09 14:17:01.205489 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:17:01.205460 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-22g4g_623826de-9ff7-4fdb-a68d-dde99e7babb2/istio-proxy/0.log" Jul 09 14:17:01.265646 ip-10-0-142-87 kubenswrapper[2573]: I0709 14:17:01.265610 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-22g4g_623826de-9ff7-4fdb-a68d-dde99e7babb2/istio-proxy/1.log"