Jul 16 04:48:09.835737 ip-10-0-135-187 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 16 04:48:09.835753 ip-10-0-135-187 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 16 04:48:09.835762 ip-10-0-135-187 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 16 04:48:09.836179 ip-10-0-135-187 systemd[1]: Failed to start Kubernetes Kubelet. Jul 16 04:48:20.585234 ip-10-0-135-187 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 16 04:48:20.585248 ip-10-0-135-187 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot a9a532e5510849e5aae5fa52c55ffabc -- Jul 16 04:50:09.520952 ip-10-0-135-187 systemd[1]: Starting Kubernetes Kubelet... Jul 16 04:50:09.887153 ip-10-0-135-187 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 04:50:09.887153 ip-10-0-135-187 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 16 04:50:09.887153 ip-10-0-135-187 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 04:50:09.887153 ip-10-0-135-187 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 16 04:50:09.887153 ip-10-0-135-187 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 04:50:09.888049 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.887966 2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 16 04:50:09.892788 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892773 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 04:50:09.892788 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892787 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 04:50:09.892788 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892791 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 04:50:09.892885 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892794 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 04:50:09.892885 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892797 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 04:50:09.892885 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892800 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 04:50:09.892885 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892802 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 04:50:09.892885 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892806 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 04:50:09.892885 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892809 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 04:50:09.892885 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892811 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 04:50:09.892885 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892813 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 04:50:09.892885 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892816 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 04:50:09.892885 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892819 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 04:50:09.892885 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892822 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 04:50:09.892885 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892824 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 04:50:09.892885 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892827 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 04:50:09.892885 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892829 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 04:50:09.892885 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892832 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 04:50:09.892885 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892834 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 04:50:09.892885 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892837 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 04:50:09.892885 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892840 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 04:50:09.892885 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892843 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 04:50:09.892885 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892845 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 16 04:50:09.893368 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892848 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 04:50:09.893368 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892850 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 04:50:09.893368 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892853 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 04:50:09.893368 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892855 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 04:50:09.893368 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892858 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 04:50:09.893368 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892860 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 04:50:09.893368 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892862 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 04:50:09.893368 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892865 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 04:50:09.893368 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892867 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 04:50:09.893368 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892870 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 04:50:09.893368 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892872 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 04:50:09.893368 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892875 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 04:50:09.893368 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892877 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 04:50:09.893368 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892881 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 04:50:09.893368 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892885 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 04:50:09.893368 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892889 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 04:50:09.893368 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892892 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 04:50:09.893368 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892895 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 04:50:09.893368 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892898 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 04:50:09.893876 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892900 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 04:50:09.893876 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892903 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 04:50:09.893876 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892906 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 04:50:09.893876 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892909 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 04:50:09.893876 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892912 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 04:50:09.893876 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892914 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 04:50:09.893876 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892916 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 04:50:09.893876 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892919 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 04:50:09.893876 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892922 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 04:50:09.893876 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892926 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 04:50:09.893876 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892928 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 04:50:09.893876 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892931 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 04:50:09.893876 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892933 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 04:50:09.893876 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892936 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 04:50:09.893876 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892940 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 04:50:09.893876 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892943 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 04:50:09.893876 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892945 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 04:50:09.893876 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892950 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 04:50:09.893876 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892953 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 04:50:09.894342 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892956 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 04:50:09.894342 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892959 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 04:50:09.894342 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892961 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 04:50:09.894342 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892964 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 04:50:09.894342 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892967 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 04:50:09.894342 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892969 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 04:50:09.894342 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892972 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 04:50:09.894342 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892974 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 04:50:09.894342 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892977 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 04:50:09.894342 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892980 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 04:50:09.894342 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892983 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 04:50:09.894342 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892985 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 04:50:09.894342 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892988 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 04:50:09.894342 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892990 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 04:50:09.894342 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892992 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 04:50:09.894342 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892995 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 04:50:09.894342 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.892997 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 04:50:09.894342 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893000 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 04:50:09.894342 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893002 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 04:50:09.894342 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893005 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 04:50:09.894342 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893007 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 04:50:09.894854 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893010 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 04:50:09.894854 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893013 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 04:50:09.894854 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893015 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 04:50:09.894854 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893017 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 04:50:09.894854 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893020 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 04:50:09.894854 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893023 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 04:50:09.894854 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893025 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 04:50:09.894854 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893028 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 04:50:09.894854 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893031 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 04:50:09.894854 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893035 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 04:50:09.894854 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893038 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 04:50:09.894854 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893040 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 04:50:09.894854 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893043 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 04:50:09.894854 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893045 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 04:50:09.894854 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893048 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 04:50:09.894854 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893050 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 04:50:09.894854 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893052 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 04:50:09.894854 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893757 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 04:50:09.894854 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893765 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 04:50:09.895312 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893769 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 04:50:09.895312 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893773 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 04:50:09.895312 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893776 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 04:50:09.895312 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893780 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 04:50:09.895312 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893783 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 04:50:09.895312 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893786 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 04:50:09.895312 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893788 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 04:50:09.895312 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893791 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 04:50:09.895312 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893793 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 04:50:09.895312 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893796 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 16 04:50:09.895312 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893799 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 04:50:09.895312 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893801 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 04:50:09.895312 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893804 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 04:50:09.895312 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893806 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 04:50:09.895312 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893810 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 04:50:09.895312 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893813 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 04:50:09.895312 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893815 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 04:50:09.895312 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893818 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 04:50:09.895312 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893820 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 04:50:09.895795 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893822 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 04:50:09.895795 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893825 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 04:50:09.895795 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893827 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 04:50:09.895795 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893830 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 04:50:09.895795 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893833 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 04:50:09.895795 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893835 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 04:50:09.895795 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893838 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 04:50:09.895795 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893840 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 04:50:09.895795 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893842 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 04:50:09.895795 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893844 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 04:50:09.895795 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893847 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 04:50:09.895795 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893849 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 04:50:09.895795 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893852 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 04:50:09.895795 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893854 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 04:50:09.895795 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893857 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 04:50:09.895795 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893859 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 04:50:09.895795 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893861 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 04:50:09.895795 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893864 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 04:50:09.895795 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893867 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 04:50:09.896263 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893869 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 04:50:09.896263 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893872 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 04:50:09.896263 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893874 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 04:50:09.896263 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893877 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 04:50:09.896263 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893880 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 04:50:09.896263 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893883 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 04:50:09.896263 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893886 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 04:50:09.896263 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893889 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 04:50:09.896263 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893892 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 04:50:09.896263 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893894 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 04:50:09.896263 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893897 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 04:50:09.896263 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893899 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 04:50:09.896263 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893901 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 04:50:09.896263 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893904 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 04:50:09.896263 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893906 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 04:50:09.896263 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893909 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 04:50:09.896263 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893911 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 04:50:09.896263 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893914 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 04:50:09.896263 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893916 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 04:50:09.896263 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893918 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 04:50:09.896789 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893921 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 04:50:09.896789 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893923 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 04:50:09.896789 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893927 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 04:50:09.896789 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893930 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 04:50:09.896789 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893933 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 04:50:09.896789 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893935 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 04:50:09.896789 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893937 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 04:50:09.896789 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893940 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 04:50:09.896789 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893942 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 04:50:09.896789 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893945 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 04:50:09.896789 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893947 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 04:50:09.896789 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893950 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 04:50:09.896789 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893952 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 04:50:09.896789 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893955 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 04:50:09.896789 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893957 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 04:50:09.896789 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893960 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 04:50:09.896789 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893962 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 04:50:09.896789 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893965 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 04:50:09.896789 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893967 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 04:50:09.896789 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893969 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 04:50:09.897270 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893972 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 04:50:09.897270 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893975 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 04:50:09.897270 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893978 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 04:50:09.897270 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893980 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 04:50:09.897270 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893983 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 04:50:09.897270 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893986 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 04:50:09.897270 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893988 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 04:50:09.897270 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893991 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 04:50:09.897270 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893994 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 04:50:09.897270 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.893998 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 04:50:09.897270 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894000 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 04:50:09.897270 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894003 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 04:50:09.897270 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894006 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 04:50:09.897270 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894008 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 04:50:09.897270 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894010 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 04:50:09.897270 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894013 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 04:50:09.897270 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894023 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 04:50:09.897270 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894026 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 04:50:09.897270 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894029 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 04:50:09.897747 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894386 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 04:50:09.897747 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894391 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 04:50:09.897747 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894394 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 04:50:09.897747 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894399 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 04:50:09.897747 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894403 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 04:50:09.897747 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894406 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 04:50:09.897747 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894409 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 04:50:09.897747 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894412 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 04:50:09.897747 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894414 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 04:50:09.897747 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894417 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 04:50:09.897747 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894419 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 04:50:09.897747 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894422 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 04:50:09.897747 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894425 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 04:50:09.897747 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894427 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 04:50:09.897747 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894430 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 04:50:09.897747 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894432 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 04:50:09.897747 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894435 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 04:50:09.897747 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894438 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 04:50:09.897747 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894441 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 04:50:09.898229 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894443 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 04:50:09.898229 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894446 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 04:50:09.898229 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894448 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 04:50:09.898229 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894451 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 04:50:09.898229 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894453 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 04:50:09.898229 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894456 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 04:50:09.898229 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894458 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 04:50:09.898229 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894461 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 04:50:09.898229 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894464 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 04:50:09.898229 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894466 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 04:50:09.898229 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894468 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 04:50:09.898229 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894471 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 04:50:09.898229 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894473 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 04:50:09.898229 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894475 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 04:50:09.898229 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894478 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 04:50:09.898229 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894481 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 04:50:09.898229 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894483 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 04:50:09.898229 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894485 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 04:50:09.898229 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894488 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 04:50:09.898229 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894491 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 04:50:09.898719 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894493 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 04:50:09.898719 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894495 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 04:50:09.898719 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894498 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 04:50:09.898719 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894500 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 04:50:09.898719 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894503 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 04:50:09.898719 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894505 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 04:50:09.898719 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894507 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 04:50:09.898719 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894510 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 04:50:09.898719 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894512 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 04:50:09.898719 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894515 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 04:50:09.898719 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894517 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 04:50:09.898719 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894520 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 04:50:09.898719 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894522 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 04:50:09.898719 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894524 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 04:50:09.898719 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894528 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 04:50:09.898719 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894531 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 04:50:09.898719 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894533 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 04:50:09.898719 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894536 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 04:50:09.898719 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894538 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 04:50:09.898719 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894541 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 04:50:09.899217 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894543 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 04:50:09.899217 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894546 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 04:50:09.899217 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894549 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 04:50:09.899217 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894551 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 04:50:09.899217 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894554 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 04:50:09.899217 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894556 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 04:50:09.899217 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894558 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 04:50:09.899217 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894560 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 04:50:09.899217 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894563 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 04:50:09.899217 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894565 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 04:50:09.899217 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894568 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 04:50:09.899217 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894570 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 04:50:09.899217 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894573 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 04:50:09.899217 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894575 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 04:50:09.899217 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894578 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 04:50:09.899217 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894580 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 04:50:09.899217 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894583 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 04:50:09.899217 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894585 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 04:50:09.899217 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894587 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 04:50:09.899217 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894590 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 04:50:09.899841 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894593 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 04:50:09.899841 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894596 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 04:50:09.899841 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894598 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 04:50:09.899841 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894601 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 04:50:09.899841 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894603 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 04:50:09.899841 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894606 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 04:50:09.899841 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894608 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 04:50:09.899841 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894610 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 04:50:09.899841 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894613 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 04:50:09.899841 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894615 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 04:50:09.899841 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894618 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 04:50:09.899841 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894622 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 04:50:09.899841 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894625 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 04:50:09.899841 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894627 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 04:50:09.899841 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894630 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 04:50:09.899841 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894632 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 04:50:09.899841 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894635 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 04:50:09.899841 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894637 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 04:50:09.899841 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894639 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 04:50:09.899841 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.894642 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 16 04:50:09.900330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894723 2561 flags.go:64] FLAG: --address="0.0.0.0" Jul 16 04:50:09.900330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894732 2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 16 04:50:09.900330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894740 2561 flags.go:64] FLAG: --anonymous-auth="true" Jul 16 04:50:09.900330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894746 2561 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 16 04:50:09.900330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894753 2561 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 16 04:50:09.900330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894757 2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 16 04:50:09.900330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894762 2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 16 04:50:09.900330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894765 2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 16 04:50:09.900330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894768 2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 16 04:50:09.900330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894771 2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 16 04:50:09.900330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894775 2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 16 04:50:09.900330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894778 2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 16 04:50:09.900330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894781 2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 16 04:50:09.900330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894784 2561 flags.go:64] FLAG: --cgroup-root="" Jul 16 04:50:09.900330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894787 2561 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 16 04:50:09.900330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894790 2561 flags.go:64] FLAG: --client-ca-file="" Jul 16 04:50:09.900330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894792 2561 flags.go:64] FLAG: --cloud-provider="external" Jul 16 04:50:09.900330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894795 2561 flags.go:64] FLAG: --cluster-dns="[]" Jul 16 04:50:09.900330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894800 2561 flags.go:64] FLAG: --cluster-domain="" Jul 16 04:50:09.900330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894803 2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 16 04:50:09.900330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894805 2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 16 04:50:09.900330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894808 2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 16 04:50:09.900330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894812 2561 flags.go:64] FLAG: --container-log-max-files="5" Jul 16 04:50:09.900330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894815 2561 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 16 04:50:09.900969 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894818 2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 16 04:50:09.900969 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894821 2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 16 04:50:09.900969 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894824 2561 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 16 04:50:09.900969 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894827 2561 flags.go:64] FLAG: --contention-profiling="false" Jul 16 04:50:09.900969 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894829 2561 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 16 04:50:09.900969 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894832 2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 16 04:50:09.900969 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894835 2561 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 16 04:50:09.900969 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894838 2561 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 16 04:50:09.900969 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894841 2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 16 04:50:09.900969 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894844 2561 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 16 04:50:09.900969 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894847 2561 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 16 04:50:09.900969 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894850 2561 flags.go:64] FLAG: --enable-load-reader="false" Jul 16 04:50:09.900969 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894853 2561 flags.go:64] FLAG: --enable-server="true" Jul 16 04:50:09.900969 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894856 2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 16 04:50:09.900969 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894860 2561 flags.go:64] FLAG: --event-burst="100" Jul 16 04:50:09.900969 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894863 2561 flags.go:64] FLAG: --event-qps="50" Jul 16 04:50:09.900969 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894866 2561 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 16 04:50:09.900969 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894871 2561 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 16 04:50:09.900969 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894874 2561 flags.go:64] FLAG: --eviction-hard="" Jul 16 04:50:09.900969 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894877 2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 16 04:50:09.900969 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894880 2561 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 16 04:50:09.900969 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894883 2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 16 04:50:09.900969 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894886 2561 flags.go:64] FLAG: --eviction-soft="" Jul 16 04:50:09.900969 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894888 2561 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 16 04:50:09.900969 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894891 2561 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 16 04:50:09.901594 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894894 2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 16 04:50:09.901594 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894897 2561 flags.go:64] FLAG: --experimental-mounter-path="" Jul 16 04:50:09.901594 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894900 2561 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 16 04:50:09.901594 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894902 2561 flags.go:64] FLAG: --fail-swap-on="true" Jul 16 04:50:09.901594 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894908 2561 flags.go:64] FLAG: --feature-gates="" Jul 16 04:50:09.901594 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894911 2561 flags.go:64] FLAG: --file-check-frequency="20s" Jul 16 04:50:09.901594 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894914 2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 16 04:50:09.901594 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894917 2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 16 04:50:09.901594 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894920 2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 16 04:50:09.901594 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894923 2561 flags.go:64] FLAG: --healthz-port="10248" Jul 16 04:50:09.901594 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894926 2561 flags.go:64] FLAG: --help="false" Jul 16 04:50:09.901594 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894929 2561 flags.go:64] FLAG: --hostname-override="ip-10-0-135-187.ec2.internal" Jul 16 04:50:09.901594 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894932 2561 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 16 04:50:09.901594 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894935 2561 flags.go:64] FLAG: --http-check-frequency="20s" Jul 16 04:50:09.901594 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894938 2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 16 04:50:09.901594 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894941 2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 16 04:50:09.901594 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894944 2561 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 16 04:50:09.901594 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894959 2561 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 16 04:50:09.901594 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894963 2561 flags.go:64] FLAG: --image-service-endpoint="" Jul 16 04:50:09.901594 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894966 2561 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 16 04:50:09.901594 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894969 2561 flags.go:64] FLAG: --kube-api-burst="100" Jul 16 04:50:09.901594 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894973 2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 16 04:50:09.901594 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894976 2561 flags.go:64] FLAG: --kube-api-qps="50" Jul 16 04:50:09.901594 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894979 2561 flags.go:64] FLAG: --kube-reserved="" Jul 16 04:50:09.902215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894982 2561 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 16 04:50:09.902215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894985 2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 16 04:50:09.902215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894988 2561 flags.go:64] FLAG: --kubelet-cgroups="" Jul 16 04:50:09.902215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894991 2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 16 04:50:09.902215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894994 2561 flags.go:64] FLAG: --lock-file="" Jul 16 04:50:09.902215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.894997 2561 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 16 04:50:09.902215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895000 2561 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 16 04:50:09.902215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895003 2561 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 16 04:50:09.902215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895008 2561 flags.go:64] FLAG: --log-json-split-stream="false" Jul 16 04:50:09.902215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895011 2561 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 16 04:50:09.902215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895013 2561 flags.go:64] FLAG: --log-text-split-stream="false" Jul 16 04:50:09.902215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895016 2561 flags.go:64] FLAG: --logging-format="text" Jul 16 04:50:09.902215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895021 2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 16 04:50:09.902215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895025 2561 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 16 04:50:09.902215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895027 2561 flags.go:64] FLAG: --manifest-url="" Jul 16 04:50:09.902215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895030 2561 flags.go:64] FLAG: --manifest-url-header="" Jul 16 04:50:09.902215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895034 2561 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 16 04:50:09.902215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895037 2561 flags.go:64] FLAG: --max-open-files="1000000" Jul 16 04:50:09.902215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895041 2561 flags.go:64] FLAG: --max-pods="110" Jul 16 04:50:09.902215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895044 2561 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 16 04:50:09.902215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895046 2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 16 04:50:09.902215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895049 2561 flags.go:64] FLAG: --memory-manager-policy="None" Jul 16 04:50:09.902215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895052 2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 16 04:50:09.902215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895055 2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 16 04:50:09.902215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895058 2561 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 16 04:50:09.902835 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895060 2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 16 04:50:09.902835 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895067 2561 flags.go:64] FLAG: --node-status-max-images="50" Jul 16 04:50:09.902835 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895070 2561 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 16 04:50:09.902835 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895073 2561 flags.go:64] FLAG: --oom-score-adj="-999" Jul 16 04:50:09.902835 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895076 2561 flags.go:64] FLAG: --pod-cidr="" Jul 16 04:50:09.902835 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895079 2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 16 04:50:09.902835 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895085 2561 flags.go:64] FLAG: --pod-manifest-path="" Jul 16 04:50:09.902835 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895087 2561 flags.go:64] FLAG: --pod-max-pids="-1" Jul 16 04:50:09.902835 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895092 2561 flags.go:64] FLAG: --pods-per-core="0" Jul 16 04:50:09.902835 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895094 2561 flags.go:64] FLAG: --port="10250" Jul 16 04:50:09.902835 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895097 2561 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 16 04:50:09.902835 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895100 2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ae6707ce67019ed3" Jul 16 04:50:09.902835 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895103 2561 flags.go:64] FLAG: --qos-reserved="" Jul 16 04:50:09.902835 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895106 2561 flags.go:64] FLAG: --read-only-port="10255" Jul 16 04:50:09.902835 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895109 2561 flags.go:64] FLAG: --register-node="true" Jul 16 04:50:09.902835 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895111 2561 flags.go:64] FLAG: --register-with-taints="" Jul 16 04:50:09.902835 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895115 2561 flags.go:64] FLAG: --registry-burst="10" Jul 16 04:50:09.902835 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895118 2561 flags.go:64] FLAG: --registry-qps="5" Jul 16 04:50:09.902835 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895120 2561 flags.go:64] FLAG: --reserved-cpus="" Jul 16 04:50:09.902835 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895124 2561 flags.go:64] FLAG: --reserved-memory="" Jul 16 04:50:09.902835 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895127 2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 16 04:50:09.902835 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895130 2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 16 04:50:09.902835 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895133 2561 flags.go:64] FLAG: --rotate-certificates="false" Jul 16 04:50:09.902835 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895136 2561 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 16 04:50:09.902835 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895139 2561 flags.go:64] FLAG: --runonce="false" Jul 16 04:50:09.903441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895142 2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 16 04:50:09.903441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895145 2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 16 04:50:09.903441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895148 2561 flags.go:64] FLAG: --seccomp-default="false" Jul 16 04:50:09.903441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895150 2561 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 16 04:50:09.903441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895153 2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 16 04:50:09.903441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895156 2561 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 16 04:50:09.903441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895159 2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 16 04:50:09.903441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895162 2561 flags.go:64] FLAG: --storage-driver-password="root" Jul 16 04:50:09.903441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895164 2561 flags.go:64] FLAG: --storage-driver-secure="false" Jul 16 04:50:09.903441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895167 2561 flags.go:64] FLAG: --storage-driver-table="stats" Jul 16 04:50:09.903441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895170 2561 flags.go:64] FLAG: --storage-driver-user="root" Jul 16 04:50:09.903441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895172 2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 16 04:50:09.903441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895175 2561 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 16 04:50:09.903441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895178 2561 flags.go:64] FLAG: --system-cgroups="" Jul 16 04:50:09.903441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895181 2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 16 04:50:09.903441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895187 2561 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 16 04:50:09.903441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895190 2561 flags.go:64] FLAG: --tls-cert-file="" Jul 16 04:50:09.903441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895193 2561 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 16 04:50:09.903441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895197 2561 flags.go:64] FLAG: --tls-min-version="" Jul 16 04:50:09.903441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895199 2561 flags.go:64] FLAG: --tls-private-key-file="" Jul 16 04:50:09.903441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895202 2561 flags.go:64] FLAG: --topology-manager-policy="none" Jul 16 04:50:09.903441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895205 2561 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 16 04:50:09.903441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895207 2561 flags.go:64] FLAG: --topology-manager-scope="container" Jul 16 04:50:09.903441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895210 2561 flags.go:64] FLAG: --v="2" Jul 16 04:50:09.903441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895214 2561 flags.go:64] FLAG: --version="false" Jul 16 04:50:09.904066 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895218 2561 flags.go:64] FLAG: --vmodule="" Jul 16 04:50:09.904066 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895223 2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 16 04:50:09.904066 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.895226 2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 16 04:50:09.904066 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895336 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 04:50:09.904066 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895339 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 04:50:09.904066 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895342 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 04:50:09.904066 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895345 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 04:50:09.904066 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895350 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 04:50:09.904066 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895353 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 04:50:09.904066 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895356 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 04:50:09.904066 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895359 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 04:50:09.904066 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895361 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 04:50:09.904066 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895364 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 04:50:09.904066 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895366 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 04:50:09.904066 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895369 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 04:50:09.904066 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895371 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 16 04:50:09.904066 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895374 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 04:50:09.904066 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895376 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 04:50:09.904066 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895379 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 04:50:09.904066 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895381 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 04:50:09.904582 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895384 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 04:50:09.904582 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895387 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 04:50:09.904582 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895391 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 04:50:09.904582 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895393 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 04:50:09.904582 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895396 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 04:50:09.904582 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895398 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 04:50:09.904582 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895401 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 04:50:09.904582 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895403 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 04:50:09.904582 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895406 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 04:50:09.904582 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895408 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 04:50:09.904582 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895411 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 04:50:09.904582 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895413 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 04:50:09.904582 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895416 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 04:50:09.904582 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895420 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 04:50:09.904582 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895422 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 04:50:09.904582 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895425 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 04:50:09.904582 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895427 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 04:50:09.904582 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895429 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 04:50:09.904582 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895432 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 04:50:09.904582 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895434 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 04:50:09.905102 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895436 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 04:50:09.905102 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895439 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 04:50:09.905102 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895441 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 04:50:09.905102 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895444 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 04:50:09.905102 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895446 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 04:50:09.905102 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895448 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 04:50:09.905102 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895451 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 04:50:09.905102 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895453 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 04:50:09.905102 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895456 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 04:50:09.905102 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895458 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 04:50:09.905102 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895461 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 04:50:09.905102 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895463 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 04:50:09.905102 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895466 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 04:50:09.905102 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895468 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 04:50:09.905102 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895472 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 04:50:09.905102 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895474 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 04:50:09.905102 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895478 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 04:50:09.905102 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895480 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 04:50:09.905102 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895482 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 04:50:09.905102 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895485 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 04:50:09.905605 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895487 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 04:50:09.905605 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895490 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 04:50:09.905605 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895492 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 04:50:09.905605 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895494 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 04:50:09.905605 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895497 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 04:50:09.905605 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895500 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 04:50:09.905605 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895503 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 04:50:09.905605 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895505 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 04:50:09.905605 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895509 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 04:50:09.905605 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895512 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 04:50:09.905605 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895515 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 04:50:09.905605 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895517 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 04:50:09.905605 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895520 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 04:50:09.905605 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895522 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 04:50:09.905605 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895524 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 04:50:09.905605 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895527 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 04:50:09.905605 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895529 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 04:50:09.905605 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895532 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 04:50:09.905605 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895534 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 04:50:09.905605 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895536 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 04:50:09.906207 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895539 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 04:50:09.906207 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895541 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 04:50:09.906207 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895544 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 04:50:09.906207 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895546 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 04:50:09.906207 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895548 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 04:50:09.906207 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895551 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 04:50:09.906207 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895554 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 04:50:09.906207 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895557 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 04:50:09.906207 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895560 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 04:50:09.906207 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895562 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 04:50:09.906207 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895565 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 04:50:09.906207 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895567 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 04:50:09.906207 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895571 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 04:50:09.906207 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895574 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 04:50:09.906207 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895577 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 04:50:09.906207 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895580 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 04:50:09.906207 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895583 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 04:50:09.906207 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895587 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 04:50:09.906207 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895590 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 04:50:09.906693 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895592 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 04:50:09.906693 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895595 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 04:50:09.906693 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.895597 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 04:50:09.906693 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.896348 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 04:50:09.906693 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.902641 2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 16 04:50:09.906693 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.902654 2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 16 04:50:09.906693 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.902673 2561 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 16 04:50:09.906693 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.902678 2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 16 04:50:09.906693 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902752 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 04:50:09.906693 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902757 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 04:50:09.906693 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902760 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 04:50:09.906693 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902763 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 04:50:09.906693 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902766 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 04:50:09.906693 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902769 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 04:50:09.906693 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902772 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 04:50:09.907071 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902775 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 04:50:09.907071 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902778 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 04:50:09.907071 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902781 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 04:50:09.907071 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902784 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 04:50:09.907071 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902786 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 04:50:09.907071 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902789 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 04:50:09.907071 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902791 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 04:50:09.907071 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902794 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 04:50:09.907071 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902796 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 04:50:09.907071 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902799 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 04:50:09.907071 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902801 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 04:50:09.907071 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902804 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 04:50:09.907071 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902806 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 04:50:09.907071 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902809 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 04:50:09.907071 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902811 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 04:50:09.907071 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902814 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 04:50:09.907071 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902817 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 04:50:09.907071 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902819 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 04:50:09.907071 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902821 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 04:50:09.907071 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902823 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 04:50:09.907549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902826 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 04:50:09.907549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902828 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 04:50:09.907549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902833 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 04:50:09.907549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902835 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 04:50:09.907549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902838 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 04:50:09.907549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902840 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 04:50:09.907549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902843 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 04:50:09.907549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902846 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 04:50:09.907549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902848 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 04:50:09.907549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902851 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 04:50:09.907549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902854 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 04:50:09.907549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902856 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 04:50:09.907549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902859 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 04:50:09.907549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902861 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 04:50:09.907549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902863 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 04:50:09.907549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902866 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 04:50:09.907549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902868 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 04:50:09.907549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902870 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 04:50:09.907549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902872 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 04:50:09.907549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902875 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 04:50:09.908057 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902877 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 04:50:09.908057 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902880 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 04:50:09.908057 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902882 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 04:50:09.908057 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902885 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 04:50:09.908057 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902887 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 04:50:09.908057 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902890 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 04:50:09.908057 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902892 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 04:50:09.908057 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902894 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 04:50:09.908057 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902897 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 04:50:09.908057 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902899 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 04:50:09.908057 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902903 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 04:50:09.908057 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902906 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 04:50:09.908057 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902909 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 04:50:09.908057 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902912 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 04:50:09.908057 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902915 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 04:50:09.908057 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902920 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 04:50:09.908057 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902923 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 04:50:09.908057 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902926 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 04:50:09.908057 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902928 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 04:50:09.908057 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902931 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 04:50:09.908549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902934 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 04:50:09.908549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902936 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 04:50:09.908549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902939 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 04:50:09.908549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902941 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 04:50:09.908549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902943 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 04:50:09.908549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902946 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 04:50:09.908549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902948 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 04:50:09.908549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902951 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 04:50:09.908549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902953 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 04:50:09.908549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902955 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 04:50:09.908549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902958 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 16 04:50:09.908549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902960 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 04:50:09.908549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902962 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 04:50:09.908549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902965 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 04:50:09.908549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902967 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 04:50:09.908549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902970 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 04:50:09.908549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902972 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 04:50:09.908549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902975 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 04:50:09.908549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902979 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 04:50:09.908549 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902982 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 04:50:09.909067 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902984 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 04:50:09.909067 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902987 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 04:50:09.909067 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902989 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 04:50:09.909067 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902991 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 04:50:09.909067 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902994 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 04:50:09.909067 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.902996 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 04:50:09.909067 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903000 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 04:50:09.909067 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903004 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 04:50:09.909067 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903007 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 04:50:09.909067 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903010 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 04:50:09.909067 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903013 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 04:50:09.909067 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903015 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 04:50:09.909067 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.903020 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 04:50:09.909067 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903154 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 04:50:09.909435 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903159 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 04:50:09.909435 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903162 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 04:50:09.909435 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903164 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 04:50:09.909435 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903167 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 04:50:09.909435 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903170 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 04:50:09.909435 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903172 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 04:50:09.909435 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903175 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 04:50:09.909435 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903177 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 04:50:09.909435 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903180 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 04:50:09.909435 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903182 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 04:50:09.909435 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903185 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 04:50:09.909435 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903187 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 04:50:09.909435 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903189 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 04:50:09.909435 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903191 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 04:50:09.909435 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903194 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 04:50:09.909435 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903196 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 04:50:09.909435 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903198 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 04:50:09.909435 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903201 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 04:50:09.909435 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903203 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 04:50:09.909435 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903205 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 04:50:09.910012 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903208 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 04:50:09.910012 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903210 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 04:50:09.910012 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903212 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 04:50:09.910012 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903214 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 04:50:09.910012 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903217 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 04:50:09.910012 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903220 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 04:50:09.910012 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903223 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 04:50:09.910012 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903225 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 04:50:09.910012 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903228 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 04:50:09.910012 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903230 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 04:50:09.910012 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903232 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 04:50:09.910012 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903234 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 04:50:09.910012 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903237 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 04:50:09.910012 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903239 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 04:50:09.910012 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903241 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 04:50:09.910012 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903244 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 04:50:09.910012 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903246 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 04:50:09.910012 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903249 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 04:50:09.910012 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903251 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 04:50:09.910012 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903253 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 16 04:50:09.910551 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903256 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 04:50:09.910551 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903258 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 04:50:09.910551 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903260 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 04:50:09.910551 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903263 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 04:50:09.910551 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903265 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 04:50:09.910551 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903267 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 04:50:09.910551 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903270 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 04:50:09.910551 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903272 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 04:50:09.910551 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903275 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 04:50:09.910551 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903277 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 04:50:09.910551 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903279 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 04:50:09.910551 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903282 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 04:50:09.910551 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903284 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 04:50:09.910551 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903287 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 04:50:09.910551 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903289 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 04:50:09.910551 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903291 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 04:50:09.910551 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903294 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 04:50:09.910551 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903296 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 04:50:09.910551 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903299 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 04:50:09.910551 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903301 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 04:50:09.911085 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903304 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 04:50:09.911085 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903307 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 04:50:09.911085 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903311 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 04:50:09.911085 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903313 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 04:50:09.911085 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903316 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 04:50:09.911085 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903318 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 04:50:09.911085 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903321 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 04:50:09.911085 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903324 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 04:50:09.911085 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903327 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 04:50:09.911085 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903330 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 04:50:09.911085 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903332 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 04:50:09.911085 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903334 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 04:50:09.911085 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903336 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 04:50:09.911085 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903339 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 04:50:09.911085 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903341 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 04:50:09.911085 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903344 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 04:50:09.911085 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903346 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 04:50:09.911085 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903348 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 04:50:09.911085 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903351 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 04:50:09.911566 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903353 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 04:50:09.911566 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903357 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 04:50:09.911566 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903360 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 04:50:09.911566 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903363 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 04:50:09.911566 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903365 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 04:50:09.911566 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903368 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 04:50:09.911566 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903370 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 04:50:09.911566 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903373 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 04:50:09.911566 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903375 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 04:50:09.911566 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903379 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 04:50:09.911566 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903381 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 04:50:09.911566 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903384 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 04:50:09.911566 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903386 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 04:50:09.911566 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903390 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 04:50:09.911566 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903394 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 04:50:09.911566 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903396 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 04:50:09.911566 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903399 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 04:50:09.911566 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903401 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 04:50:09.911566 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:09.903403 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 04:50:09.912168 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.903408 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 04:50:09.912168 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.903529 2561 server.go:962] "Client rotation is on, will bootstrap in background" Jul 16 04:50:09.912168 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.905355 2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 16 04:50:09.912168 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.907202 2561 server.go:1019] "Starting client certificate rotation" Jul 16 04:50:09.912168 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.907306 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 04:50:09.912168 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.907366 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 04:50:09.912168 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.909423 2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 04:50:09.915104 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.915080 2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 04:50:09.926586 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.926570 2561 log.go:25] "Validated CRI v1 runtime API" Jul 16 04:50:09.928441 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.928427 2561 log.go:25] "Validated CRI v1 image API" Jul 16 04:50:09.929251 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.929236 2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 16 04:50:09.932464 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.932434 2561 fs.go:135] Filesystem UUIDs: map[6bb5c1b5-72d9-4fd3-b3ce-d70cb08c29af:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 846e9eaa-7557-48c4-bc19-61dbcd2a8be9:/dev/nvme0n1p4] Jul 16 04:50:09.932554 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.932461 2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 16 04:50:09.937108 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.936996 2561 manager.go:216] Machine: {Timestamp:2026-07-16 04:50:09.935995598 +0000 UTC m=+0.308118805 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3061122 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23f1af8c8ae0a8796586654a824545 SystemUUID:ec23f1af-8c8a-e0a8-7965-86654a824545 BootID:a9a532e5-5108-49e5-aae5-fa52c55ffabc Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ae:6a:28:af:13 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ae:6a:28:af:13 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:af:14:45:eb:c4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 16 04:50:09.937108 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.937099 2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 16 04:50:09.937220 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.937192 2561 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 16 04:50:09.938131 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.938115 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 04:50:09.938654 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.938630 2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 16 04:50:09.938798 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.938658 2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-187.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 16 04:50:09.939248 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.939238 2561 topology_manager.go:138] "Creating topology manager with none policy" Jul 16 04:50:09.939283 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.939251 2561 container_manager_linux.go:309] "Creating device plugin manager" Jul 16 04:50:09.939283 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.939259 2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 04:50:09.939932 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.939920 2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 04:50:09.941024 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.941015 2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 16 04:50:09.943935 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.943923 2561 state_mem.go:36] "Initialized new in-memory state store" Jul 16 04:50:09.944063 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.944053 2561 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 16 04:50:09.944103 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.944082 2561 kubelet.go:486] "Attempting to sync node with API server" Jul 16 04:50:09.944103 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.944100 2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 16 04:50:09.944174 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.944110 2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 16 04:50:09.944174 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.944120 2561 kubelet.go:398] "Adding apiserver pod source" Jul 16 04:50:09.944174 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.944129 2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 16 04:50:09.944531 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.944517 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 04:50:09.944578 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.944537 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 04:50:09.946286 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.946270 2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 16 04:50:09.947928 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.947912 2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 16 04:50:09.948000 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.947942 2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 16 04:50:09.948085 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.948075 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 16 04:50:09.948114 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.948090 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 16 04:50:09.948114 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.948100 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 16 04:50:09.948114 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.948106 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 16 04:50:09.948114 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.948114 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 16 04:50:09.948216 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.948120 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 16 04:50:09.948644 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.948634 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 16 04:50:09.948676 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.948647 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 16 04:50:09.948676 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.948656 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 16 04:50:09.948676 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.948664 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 16 04:50:09.948781 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.948699 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 16 04:50:09.948781 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.948709 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 16 04:50:09.950086 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.950074 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 16 04:50:09.950086 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.950086 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 16 04:50:09.953261 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.953246 2561 server.go:1268] "Started kubelet" Jul 16 04:50:09.955380 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.955315 2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 16 04:50:09.955481 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.955401 2561 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 16 04:50:09.955916 ip-10-0-135-187 systemd[1]: Started Kubernetes Kubelet. Jul 16 04:50:09.956061 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.955993 2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-187.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 16 04:50:09.956152 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.953364 2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 16 04:50:09.956570 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:09.956320 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 16 04:50:09.956728 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.956669 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tl54r" Jul 16 04:50:09.957107 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.957091 2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 16 04:50:09.957319 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:09.957279 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-187.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 16 04:50:09.959331 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.959313 2561 server.go:310] "Adding debug handlers to kubelet server" Jul 16 04:50:09.961770 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.961752 2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 16 04:50:09.961872 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.961857 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 16 04:50:09.961951 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:09.961866 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-187.ec2.internal\" not found" Jul 16 04:50:09.962085 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.962048 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tl54r" Jul 16 04:50:09.962166 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:09.961875 2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 16 04:50:09.962363 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.962348 2561 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 16 04:50:09.962475 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.962460 2561 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 16 04:50:09.962563 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.962378 2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 16 04:50:09.962563 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.962548 2561 reconstruct.go:97] "Volume reconstruction finished" Jul 16 04:50:09.962563 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.962555 2561 reconciler.go:29] "Reconciler: start to sync state" Jul 16 04:50:09.962872 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.962855 2561 factory.go:153] Registering CRI-O factory Jul 16 04:50:09.962872 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.962874 2561 factory.go:223] Registration of the crio container factory successfully Jul 16 04:50:09.963012 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.962935 2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 16 04:50:09.963012 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.962942 2561 factory.go:55] Registering systemd factory Jul 16 04:50:09.963012 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.962948 2561 factory.go:223] Registration of the systemd container factory successfully Jul 16 04:50:09.963012 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.962980 2561 factory.go:103] Registering Raw factory Jul 16 04:50:09.963012 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.962989 2561 manager.go:1196] Started watching for new ooms in manager Jul 16 04:50:09.963387 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.963262 2561 manager.go:349] Starting recovery of all containers Jul 16 04:50:09.963755 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:09.962876 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-187.ec2.internal.18c2abe6c089b7bd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-187.ec2.internal,UID:ip-10-0-135-187.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-187.ec2.internal,},FirstTimestamp:2026-07-16 04:50:09.953216445 +0000 UTC m=+0.325339642,LastTimestamp:2026-07-16 04:50:09.953216445 +0000 UTC m=+0.325339642,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-187.ec2.internal,}" Jul 16 04:50:09.968063 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.967884 2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 04:50:09.970833 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:09.970806 2561 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-187.ec2.internal\" not found" node="ip-10-0-135-187.ec2.internal" Jul 16 04:50:09.972440 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.972425 2561 manager.go:354] Recovery completed Jul 16 04:50:09.977892 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.977877 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 04:50:09.980729 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.980714 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-187.ec2.internal" event="NodeHasSufficientMemory" Jul 16 04:50:09.980802 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.980742 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 04:50:09.980802 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.980752 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-187.ec2.internal" event="NodeHasSufficientPID" Jul 16 04:50:09.981256 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.981245 2561 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 16 04:50:09.981307 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.981257 2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 16 04:50:09.981307 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.981272 2561 state_mem.go:36] "Initialized new in-memory state store" Jul 16 04:50:09.981394 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.981320 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 04:50:09.982333 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.982322 2561 policy_none.go:49] "None policy: Start" Jul 16 04:50:09.982370 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.982337 2561 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 16 04:50:09.982370 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.982347 2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 16 04:50:09.983161 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.983151 2561 policy_none.go:47] "Start" Jul 16 04:50:09.983962 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.983946 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-187.ec2.internal" event="NodeHasSufficientMemory" Jul 16 04:50:09.984043 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.983971 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 04:50:09.984043 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:09.983981 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-187.ec2.internal" event="NodeHasSufficientPID" Jul 16 04:50:10.023259 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.023243 2561 manager.go:337] "Starting Device Plugin manager" Jul 16 04:50:10.023351 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:10.023275 2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 16 04:50:10.023351 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.023287 2561 server.go:87] "Starting device plugin registration server" Jul 16 04:50:10.023522 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.023509 2561 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 16 04:50:10.023585 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.023524 2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 16 04:50:10.023638 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.023614 2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 16 04:50:10.023726 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.023711 2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 16 04:50:10.023726 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.023726 2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 16 04:50:10.024211 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:10.024188 2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 16 04:50:10.024292 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:10.024245 2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-187.ec2.internal\" not found" Jul 16 04:50:10.093032 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.093008 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 16 04:50:10.094339 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.094323 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 16 04:50:10.094400 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.094366 2561 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 16 04:50:10.094400 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.094390 2561 kubelet.go:2448] "Starting kubelet main sync loop" Jul 16 04:50:10.094471 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:10.094426 2561 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 16 04:50:10.096547 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.096525 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 04:50:10.124360 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.124342 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 04:50:10.125100 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.125084 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-187.ec2.internal" event="NodeHasSufficientMemory" Jul 16 04:50:10.125171 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.125115 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 04:50:10.125171 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.125129 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-187.ec2.internal" event="NodeHasSufficientPID" Jul 16 04:50:10.125171 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.125149 2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-187.ec2.internal" Jul 16 04:50:10.132880 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.132865 2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-187.ec2.internal" Jul 16 04:50:10.132931 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:10.132920 2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-187.ec2.internal\": node \"ip-10-0-135-187.ec2.internal\" not found" Jul 16 04:50:10.148724 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:10.148672 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-187.ec2.internal\" not found" Jul 16 04:50:10.195436 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.195407 2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-187.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-187.ec2.internal"] Jul 16 04:50:10.195495 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.195485 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 04:50:10.197793 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.197779 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-187.ec2.internal" event="NodeHasSufficientMemory" Jul 16 04:50:10.197848 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.197806 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 04:50:10.197848 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.197821 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-187.ec2.internal" event="NodeHasSufficientPID" Jul 16 04:50:10.199172 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.199160 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 04:50:10.199315 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.199288 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-187.ec2.internal" Jul 16 04:50:10.199351 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.199330 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 04:50:10.201036 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.201022 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-187.ec2.internal" event="NodeHasSufficientMemory" Jul 16 04:50:10.201109 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.201045 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 04:50:10.201109 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.201055 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-187.ec2.internal" event="NodeHasSufficientPID" Jul 16 04:50:10.201109 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.201055 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-187.ec2.internal" event="NodeHasSufficientMemory" Jul 16 04:50:10.201109 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.201077 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 04:50:10.201109 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.201087 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-187.ec2.internal" event="NodeHasSufficientPID" Jul 16 04:50:10.202334 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.202314 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-187.ec2.internal" Jul 16 04:50:10.202410 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.202351 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 04:50:10.203045 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.203031 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-187.ec2.internal" event="NodeHasSufficientMemory" Jul 16 04:50:10.203113 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.203070 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 04:50:10.203113 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.203081 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-187.ec2.internal" event="NodeHasSufficientPID" Jul 16 04:50:10.228228 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:10.228208 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-187.ec2.internal\" not found" node="ip-10-0-135-187.ec2.internal" Jul 16 04:50:10.231346 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:10.231332 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-187.ec2.internal\" not found" node="ip-10-0-135-187.ec2.internal" Jul 16 04:50:10.248774 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:10.248755 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-187.ec2.internal\" not found" Jul 16 04:50:10.263780 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.263755 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7dae16fba94631b5028f0bd95f98f48b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-187.ec2.internal\" (UID: \"7dae16fba94631b5028f0bd95f98f48b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-187.ec2.internal" Jul 16 04:50:10.263780 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.263779 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d36fd6545199dd402a0bbf93c6fc2c71-config\") pod \"kube-apiserver-proxy-ip-10-0-135-187.ec2.internal\" (UID: \"d36fd6545199dd402a0bbf93c6fc2c71\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-187.ec2.internal" Jul 16 04:50:10.263900 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.263796 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7dae16fba94631b5028f0bd95f98f48b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-187.ec2.internal\" (UID: \"7dae16fba94631b5028f0bd95f98f48b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-187.ec2.internal" Jul 16 04:50:10.349171 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:10.349150 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-187.ec2.internal\" not found" Jul 16 04:50:10.364493 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.364475 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7dae16fba94631b5028f0bd95f98f48b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-187.ec2.internal\" (UID: \"7dae16fba94631b5028f0bd95f98f48b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-187.ec2.internal" Jul 16 04:50:10.364549 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.364498 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7dae16fba94631b5028f0bd95f98f48b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-187.ec2.internal\" (UID: \"7dae16fba94631b5028f0bd95f98f48b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-187.ec2.internal" Jul 16 04:50:10.364549 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.364519 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d36fd6545199dd402a0bbf93c6fc2c71-config\") pod \"kube-apiserver-proxy-ip-10-0-135-187.ec2.internal\" (UID: \"d36fd6545199dd402a0bbf93c6fc2c71\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-187.ec2.internal" Jul 16 04:50:10.364549 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.364527 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7dae16fba94631b5028f0bd95f98f48b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-187.ec2.internal\" (UID: \"7dae16fba94631b5028f0bd95f98f48b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-187.ec2.internal" Jul 16 04:50:10.364650 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.364561 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d36fd6545199dd402a0bbf93c6fc2c71-config\") pod \"kube-apiserver-proxy-ip-10-0-135-187.ec2.internal\" (UID: \"d36fd6545199dd402a0bbf93c6fc2c71\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-187.ec2.internal" Jul 16 04:50:10.364650 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.364577 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7dae16fba94631b5028f0bd95f98f48b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-187.ec2.internal\" (UID: \"7dae16fba94631b5028f0bd95f98f48b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-187.ec2.internal" Jul 16 04:50:10.449932 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:10.449878 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-187.ec2.internal\" not found" Jul 16 04:50:10.530362 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.530339 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-187.ec2.internal" Jul 16 04:50:10.534343 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.533961 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-187.ec2.internal" Jul 16 04:50:10.550389 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:10.550370 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-187.ec2.internal\" not found" Jul 16 04:50:10.650926 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:10.650903 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-187.ec2.internal\" not found" Jul 16 04:50:10.751386 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:10.751362 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-187.ec2.internal\" not found" Jul 16 04:50:10.845325 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.845299 2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 04:50:10.851664 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:10.851638 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-187.ec2.internal\" not found" Jul 16 04:50:10.906617 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.906590 2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 16 04:50:10.907237 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.906744 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 04:50:10.907237 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.906759 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 04:50:10.907237 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.906759 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 04:50:10.952384 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:10.952362 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-187.ec2.internal\" not found" Jul 16 04:50:10.965114 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.965050 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 04:45:10 +0000 UTC" deadline="2028-02-11 15:27:13.304001807 +0000 UTC" Jul 16 04:50:10.965114 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:10.965082 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13810h37m2.338922912s" Jul 16 04:50:11.042696 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:11.042658 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dae16fba94631b5028f0bd95f98f48b.slice/crio-e946ed2ac8dfa2ae0a2f1df0cc1e5c854cdc59566c78e338334016ff475acabc WatchSource:0}: Error finding container e946ed2ac8dfa2ae0a2f1df0cc1e5c854cdc59566c78e338334016ff475acabc: Status 404 returned error can't find the container with id e946ed2ac8dfa2ae0a2f1df0cc1e5c854cdc59566c78e338334016ff475acabc Jul 16 04:50:11.042954 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:11.042932 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd36fd6545199dd402a0bbf93c6fc2c71.slice/crio-44f0d45fda39b532ca50b0ea3f1e1159a903f99b560f00c5f3768be5a36038fc WatchSource:0}: Error finding container 44f0d45fda39b532ca50b0ea3f1e1159a903f99b560f00c5f3768be5a36038fc: Status 404 returned error can't find the container with id 44f0d45fda39b532ca50b0ea3f1e1159a903f99b560f00c5f3768be5a36038fc Jul 16 04:50:11.047974 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.047948 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 04:50:11.052875 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:11.052852 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-187.ec2.internal\" not found" Jul 16 04:50:11.098151 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.098109 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-187.ec2.internal" event={"ID":"d36fd6545199dd402a0bbf93c6fc2c71","Type":"ContainerStarted","Data":"44f0d45fda39b532ca50b0ea3f1e1159a903f99b560f00c5f3768be5a36038fc"} Jul 16 04:50:11.099356 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.099330 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-187.ec2.internal" event={"ID":"7dae16fba94631b5028f0bd95f98f48b","Type":"ContainerStarted","Data":"e946ed2ac8dfa2ae0a2f1df0cc1e5c854cdc59566c78e338334016ff475acabc"} Jul 16 04:50:11.109436 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.109403 2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 04:50:11.162226 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.162198 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-187.ec2.internal" Jul 16 04:50:11.173903 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.173879 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 04:50:11.175206 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.175191 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-187.ec2.internal" Jul 16 04:50:11.185055 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.185031 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 04:50:11.736068 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.736032 2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 04:50:11.814805 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.814759 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 04:50:11.945865 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.945838 2561 apiserver.go:52] "Watching apiserver" Jul 16 04:50:11.954789 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.954747 2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 16 04:50:11.955259 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.955230 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-phkqm","openshift-ovn-kubernetes/ovnkube-node-bsqdp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht","openshift-cluster-node-tuning-operator/tuned-ww44m","openshift-image-registry/node-ca-2lr5q","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-187.ec2.internal","openshift-multus/multus-additional-cni-plugins-hkq2b","openshift-multus/network-metrics-daemon-2srzm","openshift-network-diagnostics/network-check-target-jbmmm","openshift-network-operator/iptables-alerter-n7ljg","kube-system/global-pull-secret-syncer-p97p6","kube-system/konnectivity-agent-9lpb5","kube-system/kube-apiserver-proxy-ip-10-0-135-187.ec2.internal","openshift-dns/node-resolver-vkqpj"] Jul 16 04:50:11.957824 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.957757 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:11.960406 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.960383 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:11.960532 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:11.960453 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p97p6" podUID="67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c" Jul 16 04:50:11.960914 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.960879 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 16 04:50:11.961033 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.960946 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 16 04:50:11.961139 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.961120 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 16 04:50:11.961200 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.961151 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 16 04:50:11.961323 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.961306 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 16 04:50:11.961398 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.961315 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dbp4v\"" Jul 16 04:50:11.963068 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.963047 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 16 04:50:11.965391 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.964989 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" Jul 16 04:50:11.967864 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.967849 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 16 04:50:11.967940 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.967865 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 16 04:50:11.967993 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.967980 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 16 04:50:11.968116 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.968100 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-875rr\"" Jul 16 04:50:11.968151 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.968130 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:11.970185 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.970161 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2lr5q" Jul 16 04:50:11.971273 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.971254 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 16 04:50:11.971375 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.971313 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 16 04:50:11.971375 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.971253 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ghc4r\"" Jul 16 04:50:11.972366 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.972336 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vkqpj" Jul 16 04:50:11.972864 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.972844 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 16 04:50:11.972955 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.972904 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 16 04:50:11.973018 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.972844 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 16 04:50:11.973139 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.973122 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5wlqb\"" Jul 16 04:50:11.974898 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.974856 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e18af1c7-b01c-40d1-825b-65cb623e4c5d-cnibin\") pod \"multus-additional-cni-plugins-hkq2b\" (UID: \"e18af1c7-b01c-40d1-825b-65cb623e4c5d\") " pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:11.975222 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.975194 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5fsvh\"" Jul 16 04:50:11.975421 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.975401 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 16 04:50:11.975501 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.975467 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 16 04:50:11.975956 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.975406 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:11.976049 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.974924 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e18af1c7-b01c-40d1-825b-65cb623e4c5d-cni-binary-copy\") pod \"multus-additional-cni-plugins-hkq2b\" (UID: \"e18af1c7-b01c-40d1-825b-65cb623e4c5d\") " pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:11.976112 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:11.976045 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbmmm" podUID="0f651332-08d0-4eb5-b1e8-5d9437727895" Jul 16 04:50:11.976112 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.976072 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-original-pull-secret\") pod \"global-pull-secret-syncer-p97p6\" (UID: \"67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c\") " pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:11.976112 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.976105 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d575250d-2ab4-457d-a702-4386d68d44a3-socket-dir\") pod \"aws-ebs-csi-driver-node-zn4ht\" (UID: \"d575250d-2ab4-457d-a702-4386d68d44a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" Jul 16 04:50:11.976255 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.976155 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d575250d-2ab4-457d-a702-4386d68d44a3-registration-dir\") pod \"aws-ebs-csi-driver-node-zn4ht\" (UID: \"d575250d-2ab4-457d-a702-4386d68d44a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" Jul 16 04:50:11.976255 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.976191 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e18af1c7-b01c-40d1-825b-65cb623e4c5d-os-release\") pod \"multus-additional-cni-plugins-hkq2b\" (UID: \"e18af1c7-b01c-40d1-825b-65cb623e4c5d\") " pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:11.976255 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.976227 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e18af1c7-b01c-40d1-825b-65cb623e4c5d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hkq2b\" (UID: \"e18af1c7-b01c-40d1-825b-65cb623e4c5d\") " pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:11.976399 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.976266 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e18af1c7-b01c-40d1-825b-65cb623e4c5d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hkq2b\" (UID: \"e18af1c7-b01c-40d1-825b-65cb623e4c5d\") " pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:11.976399 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.976297 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-kubelet-config\") pod \"global-pull-secret-syncer-p97p6\" (UID: \"67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c\") " pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:11.976399 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.976333 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d575250d-2ab4-457d-a702-4386d68d44a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zn4ht\" (UID: \"d575250d-2ab4-457d-a702-4386d68d44a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" Jul 16 04:50:11.976399 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.976369 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d575250d-2ab4-457d-a702-4386d68d44a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-zn4ht\" (UID: \"d575250d-2ab4-457d-a702-4386d68d44a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" Jul 16 04:50:11.976595 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.976399 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28zj2\" (UniqueName: \"kubernetes.io/projected/d575250d-2ab4-457d-a702-4386d68d44a3-kube-api-access-28zj2\") pod \"aws-ebs-csi-driver-node-zn4ht\" (UID: \"d575250d-2ab4-457d-a702-4386d68d44a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" Jul 16 04:50:11.976595 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.976475 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e18af1c7-b01c-40d1-825b-65cb623e4c5d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hkq2b\" (UID: \"e18af1c7-b01c-40d1-825b-65cb623e4c5d\") " pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:11.976595 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.976518 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d575250d-2ab4-457d-a702-4386d68d44a3-sys-fs\") pod \"aws-ebs-csi-driver-node-zn4ht\" (UID: \"d575250d-2ab4-457d-a702-4386d68d44a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" Jul 16 04:50:11.976735 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.976707 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e18af1c7-b01c-40d1-825b-65cb623e4c5d-system-cni-dir\") pod \"multus-additional-cni-plugins-hkq2b\" (UID: \"e18af1c7-b01c-40d1-825b-65cb623e4c5d\") " pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:11.976808 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.976772 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf5rm\" (UniqueName: \"kubernetes.io/projected/e18af1c7-b01c-40d1-825b-65cb623e4c5d-kube-api-access-xf5rm\") pod \"multus-additional-cni-plugins-hkq2b\" (UID: \"e18af1c7-b01c-40d1-825b-65cb623e4c5d\") " pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:11.976921 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.976808 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-dbus\") pod \"global-pull-secret-syncer-p97p6\" (UID: \"67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c\") " pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:11.976921 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.976900 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 04:50:11.977033 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.976939 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d575250d-2ab4-457d-a702-4386d68d44a3-device-dir\") pod \"aws-ebs-csi-driver-node-zn4ht\" (UID: \"d575250d-2ab4-457d-a702-4386d68d44a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" Jul 16 04:50:11.979810 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.979790 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:11.979920 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:11.979850 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2srzm" podUID="8b4ec3ce-70fc-4716-a3c6-5ab123a8be45" Jul 16 04:50:11.982491 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.982471 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:11.984932 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.984910 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-n7ljg" Jul 16 04:50:11.985318 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.985293 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 16 04:50:11.985580 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.985559 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 16 04:50:11.985680 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.985655 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 16 04:50:11.985818 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.985710 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 16 04:50:11.986063 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.986030 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hmdbw\"" Jul 16 04:50:11.986288 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.986242 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 16 04:50:11.987149 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.987030 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 16 04:50:11.988013 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.987993 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 16 04:50:11.988123 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.988112 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 16 04:50:11.988323 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.988306 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-phkqm" Jul 16 04:50:11.988637 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.988620 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mknws\"" Jul 16 04:50:11.988833 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.988813 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 16 04:50:11.990048 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.990026 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9lpb5" Jul 16 04:50:11.991197 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.991175 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 16 04:50:11.991294 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.991183 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wgql7\"" Jul 16 04:50:11.992506 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.992455 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 16 04:50:11.992841 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.992821 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 16 04:50:11.992939 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:11.992843 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rtl9h\"" Jul 16 04:50:12.000719 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.000589 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9ltgs" Jul 16 04:50:12.008001 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.007979 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9ltgs" Jul 16 04:50:12.063797 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.063775 2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 16 04:50:12.077172 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077149 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ffc2facd-a805-43cc-91d2-e35ffff459fd-host\") pod \"node-ca-2lr5q\" (UID: \"ffc2facd-a805-43cc-91d2-e35ffff459fd\") " pod="openshift-image-registry/node-ca-2lr5q" Jul 16 04:50:12.077279 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077178 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-cnibin\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.077279 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077203 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e18af1c7-b01c-40d1-825b-65cb623e4c5d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hkq2b\" (UID: \"e18af1c7-b01c-40d1-825b-65cb623e4c5d\") " pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:12.077279 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077221 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e18af1c7-b01c-40d1-825b-65cb623e4c5d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hkq2b\" (UID: \"e18af1c7-b01c-40d1-825b-65cb623e4c5d\") " pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:12.077279 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077243 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-run-ovn\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.077279 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077267 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-host-run-ovn-kubernetes\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.077523 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077290 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99688117-4206-4fca-be31-0ef08f532483-iptables-alerter-script\") pod \"iptables-alerter-n7ljg\" (UID: \"99688117-4206-4fca-be31-0ef08f532483\") " pod="openshift-network-operator/iptables-alerter-n7ljg" Jul 16 04:50:12.077523 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077306 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5dtb\" (UniqueName: \"kubernetes.io/projected/ffc2facd-a805-43cc-91d2-e35ffff459fd-kube-api-access-f5dtb\") pod \"node-ca-2lr5q\" (UID: \"ffc2facd-a805-43cc-91d2-e35ffff459fd\") " pod="openshift-image-registry/node-ca-2lr5q" Jul 16 04:50:12.077523 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077346 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-host-run-k8s-cni-cncf-io\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.077523 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077373 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-host-var-lib-cni-multus\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.077523 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077389 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/176b1c0f-1174-47ef-aa5c-25e699c63e02-ovn-node-metrics-cert\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.077523 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077406 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqfbb\" (UniqueName: \"kubernetes.io/projected/c40a4006-55d3-4359-b554-d71da361a5eb-kube-api-access-sqfbb\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.077523 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077460 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e18af1c7-b01c-40d1-825b-65cb623e4c5d-system-cni-dir\") pod \"multus-additional-cni-plugins-hkq2b\" (UID: \"e18af1c7-b01c-40d1-825b-65cb623e4c5d\") " pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:12.077523 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077500 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-dbus\") pod \"global-pull-secret-syncer-p97p6\" (UID: \"67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c\") " pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:12.077523 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077524 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-multus-socket-dir-parent\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.077895 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077541 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/176b1c0f-1174-47ef-aa5c-25e699c63e02-env-overrides\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.077895 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077570 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-run\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.077895 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077568 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e18af1c7-b01c-40d1-825b-65cb623e4c5d-system-cni-dir\") pod \"multus-additional-cni-plugins-hkq2b\" (UID: \"e18af1c7-b01c-40d1-825b-65cb623e4c5d\") " pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:12.077895 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077602 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d575250d-2ab4-457d-a702-4386d68d44a3-registration-dir\") pod \"aws-ebs-csi-driver-node-zn4ht\" (UID: \"d575250d-2ab4-457d-a702-4386d68d44a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" Jul 16 04:50:12.077895 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077640 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-multus-conf-dir\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.077895 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077662 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d575250d-2ab4-457d-a702-4386d68d44a3-registration-dir\") pod \"aws-ebs-csi-driver-node-zn4ht\" (UID: \"d575250d-2ab4-457d-a702-4386d68d44a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" Jul 16 04:50:12.077895 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077668 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-run-openvswitch\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.077895 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077682 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-dbus\") pod \"global-pull-secret-syncer-p97p6\" (UID: \"67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c\") " pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:12.077895 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077711 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-log-socket\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.077895 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077740 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e18af1c7-b01c-40d1-825b-65cb623e4c5d-os-release\") pod \"multus-additional-cni-plugins-hkq2b\" (UID: \"e18af1c7-b01c-40d1-825b-65cb623e4c5d\") " pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:12.077895 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077766 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d575250d-2ab4-457d-a702-4386d68d44a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-zn4ht\" (UID: \"d575250d-2ab4-457d-a702-4386d68d44a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" Jul 16 04:50:12.077895 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077790 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-hostroot\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.077895 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077801 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e18af1c7-b01c-40d1-825b-65cb623e4c5d-os-release\") pod \"multus-additional-cni-plugins-hkq2b\" (UID: \"e18af1c7-b01c-40d1-825b-65cb623e4c5d\") " pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:12.077895 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077814 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmq5s\" (UniqueName: \"kubernetes.io/projected/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-kube-api-access-kmq5s\") pod \"network-metrics-daemon-2srzm\" (UID: \"8b4ec3ce-70fc-4716-a3c6-5ab123a8be45\") " pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:12.077895 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077846 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d575250d-2ab4-457d-a702-4386d68d44a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-zn4ht\" (UID: \"d575250d-2ab4-457d-a702-4386d68d44a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" Jul 16 04:50:12.077895 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077852 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/176b1c0f-1174-47ef-aa5c-25e699c63e02-ovnkube-script-lib\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.077895 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077892 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d575250d-2ab4-457d-a702-4386d68d44a3-sys-fs\") pod \"aws-ebs-csi-driver-node-zn4ht\" (UID: \"d575250d-2ab4-457d-a702-4386d68d44a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" Jul 16 04:50:12.078589 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077912 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e18af1c7-b01c-40d1-825b-65cb623e4c5d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hkq2b\" (UID: \"e18af1c7-b01c-40d1-825b-65cb623e4c5d\") " pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:12.078589 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077920 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/076bc066-399e-417e-a245-14833a939ca8-hosts-file\") pod \"node-resolver-vkqpj\" (UID: \"076bc066-399e-417e-a245-14833a939ca8\") " pod="openshift-dns/node-resolver-vkqpj" Jul 16 04:50:12.078589 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077973 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d575250d-2ab4-457d-a702-4386d68d44a3-sys-fs\") pod \"aws-ebs-csi-driver-node-zn4ht\" (UID: \"d575250d-2ab4-457d-a702-4386d68d44a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" Jul 16 04:50:12.078589 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.077988 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs\") pod \"network-metrics-daemon-2srzm\" (UID: \"8b4ec3ce-70fc-4716-a3c6-5ab123a8be45\") " pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:12.078589 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078017 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-host-cni-netd\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.078589 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078043 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-etc-kubernetes\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.078589 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078068 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-host-slash\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.078589 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078094 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-var-lib-openvswitch\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.078589 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078188 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-lib-modules\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.078589 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078229 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e18af1c7-b01c-40d1-825b-65cb623e4c5d-cni-binary-copy\") pod \"multus-additional-cni-plugins-hkq2b\" (UID: \"e18af1c7-b01c-40d1-825b-65cb623e4c5d\") " pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:12.078589 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078257 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-original-pull-secret\") pod \"global-pull-secret-syncer-p97p6\" (UID: \"67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c\") " pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:12.078589 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078284 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d575250d-2ab4-457d-a702-4386d68d44a3-socket-dir\") pod \"aws-ebs-csi-driver-node-zn4ht\" (UID: \"d575250d-2ab4-457d-a702-4386d68d44a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" Jul 16 04:50:12.078589 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078312 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq2r5\" (UniqueName: \"kubernetes.io/projected/076bc066-399e-417e-a245-14833a939ca8-kube-api-access-wq2r5\") pod \"node-resolver-vkqpj\" (UID: \"076bc066-399e-417e-a245-14833a939ca8\") " pod="openshift-dns/node-resolver-vkqpj" Jul 16 04:50:12.078589 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078338 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-host-var-lib-cni-bin\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.078589 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078355 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e18af1c7-b01c-40d1-825b-65cb623e4c5d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hkq2b\" (UID: \"e18af1c7-b01c-40d1-825b-65cb623e4c5d\") " pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:12.078589 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078362 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-host-cni-bin\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.079383 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078386 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/176b1c0f-1174-47ef-aa5c-25e699c63e02-ovnkube-config\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.079383 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:12.078406 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 04:50:12.079383 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078409 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/076bc066-399e-417e-a245-14833a939ca8-tmp-dir\") pod \"node-resolver-vkqpj\" (UID: \"076bc066-399e-417e-a245-14833a939ca8\") " pod="openshift-dns/node-resolver-vkqpj" Jul 16 04:50:12.079383 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078433 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-os-release\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.079383 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078459 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-host-run-multus-certs\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.079383 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078474 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d575250d-2ab4-457d-a702-4386d68d44a3-socket-dir\") pod \"aws-ebs-csi-driver-node-zn4ht\" (UID: \"d575250d-2ab4-457d-a702-4386d68d44a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" Jul 16 04:50:12.079383 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078482 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-systemd-units\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.079383 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:12.078523 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-original-pull-secret podName:67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c nodeName:}" failed. No retries permitted until 2026-07-16 04:50:12.578501125 +0000 UTC m=+2.950624332 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-original-pull-secret") pod "global-pull-secret-syncer-p97p6" (UID: "67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c") : object "kube-system"/"original-pull-secret" not registered Jul 16 04:50:12.079383 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078547 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-etc-openvswitch\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.079383 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078585 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-etc-sysconfig\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.079383 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078613 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-multus-cni-dir\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.079383 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078640 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-host-run-netns\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.079383 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078665 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-host-var-lib-kubelet\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.079383 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078670 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e18af1c7-b01c-40d1-825b-65cb623e4c5d-cni-binary-copy\") pod \"multus-additional-cni-plugins-hkq2b\" (UID: \"e18af1c7-b01c-40d1-825b-65cb623e4c5d\") " pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:12.079383 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078690 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngg9s\" (UniqueName: \"kubernetes.io/projected/176b1c0f-1174-47ef-aa5c-25e699c63e02-kube-api-access-ngg9s\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.079383 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078742 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-etc-sysctl-d\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.079383 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078809 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-host\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.080155 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078843 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-system-cni-dir\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.080155 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078868 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm8zs\" (UniqueName: \"kubernetes.io/projected/607bd4db-30e7-4ddb-9524-42a044c9a203-kube-api-access-xm8zs\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.080155 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078891 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-host-kubelet\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.080155 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078917 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-etc-modprobe-d\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.080155 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.078978 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-var-lib-kubelet\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.080155 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079005 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-kubelet-config\") pod \"global-pull-secret-syncer-p97p6\" (UID: \"67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c\") " pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:12.080155 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079021 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d575250d-2ab4-457d-a702-4386d68d44a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zn4ht\" (UID: \"d575250d-2ab4-457d-a702-4386d68d44a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" Jul 16 04:50:12.080155 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079045 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28zj2\" (UniqueName: \"kubernetes.io/projected/d575250d-2ab4-457d-a702-4386d68d44a3-kube-api-access-28zj2\") pod \"aws-ebs-csi-driver-node-zn4ht\" (UID: \"d575250d-2ab4-457d-a702-4386d68d44a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" Jul 16 04:50:12.080155 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079074 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.080155 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079091 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d575250d-2ab4-457d-a702-4386d68d44a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zn4ht\" (UID: \"d575250d-2ab4-457d-a702-4386d68d44a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" Jul 16 04:50:12.080155 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079101 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-sys\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.080155 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079107 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-kubelet-config\") pod \"global-pull-secret-syncer-p97p6\" (UID: \"67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c\") " pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:12.080155 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079124 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ffc2facd-a805-43cc-91d2-e35ffff459fd-serviceca\") pod \"node-ca-2lr5q\" (UID: \"ffc2facd-a805-43cc-91d2-e35ffff459fd\") " pod="openshift-image-registry/node-ca-2lr5q" Jul 16 04:50:12.080155 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079150 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e18af1c7-b01c-40d1-825b-65cb623e4c5d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hkq2b\" (UID: \"e18af1c7-b01c-40d1-825b-65cb623e4c5d\") " pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:12.080155 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079173 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/607bd4db-30e7-4ddb-9524-42a044c9a203-cni-binary-copy\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.080155 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079197 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-etc-kubernetes\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.080155 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079223 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1c5e6cb3-b901-4b3c-a82e-91aa7ccb0ecd-konnectivity-ca\") pod \"konnectivity-agent-9lpb5\" (UID: \"1c5e6cb3-b901-4b3c-a82e-91aa7ccb0ecd\") " pod="kube-system/konnectivity-agent-9lpb5" Jul 16 04:50:12.080859 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079250 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs4tn\" (UniqueName: \"kubernetes.io/projected/0f651332-08d0-4eb5-b1e8-5d9437727895-kube-api-access-gs4tn\") pod \"network-check-target-jbmmm\" (UID: \"0f651332-08d0-4eb5-b1e8-5d9437727895\") " pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:12.080859 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079272 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-node-log\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.080859 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079308 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99688117-4206-4fca-be31-0ef08f532483-host-slash\") pod \"iptables-alerter-n7ljg\" (UID: \"99688117-4206-4fca-be31-0ef08f532483\") " pod="openshift-network-operator/iptables-alerter-n7ljg" Jul 16 04:50:12.080859 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079326 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e18af1c7-b01c-40d1-825b-65cb623e4c5d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hkq2b\" (UID: \"e18af1c7-b01c-40d1-825b-65cb623e4c5d\") " pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:12.080859 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079333 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-etc-sysctl-conf\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.080859 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079393 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf5rm\" (UniqueName: \"kubernetes.io/projected/e18af1c7-b01c-40d1-825b-65cb623e4c5d-kube-api-access-xf5rm\") pod \"multus-additional-cni-plugins-hkq2b\" (UID: \"e18af1c7-b01c-40d1-825b-65cb623e4c5d\") " pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:12.080859 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079437 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d575250d-2ab4-457d-a702-4386d68d44a3-device-dir\") pod \"aws-ebs-csi-driver-node-zn4ht\" (UID: \"d575250d-2ab4-457d-a702-4386d68d44a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" Jul 16 04:50:12.080859 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079468 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1c5e6cb3-b901-4b3c-a82e-91aa7ccb0ecd-agent-certs\") pod \"konnectivity-agent-9lpb5\" (UID: \"1c5e6cb3-b901-4b3c-a82e-91aa7ccb0ecd\") " pod="kube-system/konnectivity-agent-9lpb5" Jul 16 04:50:12.080859 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079518 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d575250d-2ab4-457d-a702-4386d68d44a3-device-dir\") pod \"aws-ebs-csi-driver-node-zn4ht\" (UID: \"d575250d-2ab4-457d-a702-4386d68d44a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" Jul 16 04:50:12.080859 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079532 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-host-run-netns\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.080859 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079630 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-run-systemd\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.080859 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079648 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn45b\" (UniqueName: \"kubernetes.io/projected/99688117-4206-4fca-be31-0ef08f532483-kube-api-access-bn45b\") pod \"iptables-alerter-n7ljg\" (UID: \"99688117-4206-4fca-be31-0ef08f532483\") " pod="openshift-network-operator/iptables-alerter-n7ljg" Jul 16 04:50:12.080859 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079668 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-etc-systemd\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.080859 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079683 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c40a4006-55d3-4359-b554-d71da361a5eb-etc-tuned\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.080859 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079721 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e18af1c7-b01c-40d1-825b-65cb623e4c5d-cnibin\") pod \"multus-additional-cni-plugins-hkq2b\" (UID: \"e18af1c7-b01c-40d1-825b-65cb623e4c5d\") " pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:12.080859 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079736 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/607bd4db-30e7-4ddb-9524-42a044c9a203-multus-daemon-config\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.081383 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079808 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c40a4006-55d3-4359-b554-d71da361a5eb-tmp\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.081383 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.079862 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e18af1c7-b01c-40d1-825b-65cb623e4c5d-cnibin\") pod \"multus-additional-cni-plugins-hkq2b\" (UID: \"e18af1c7-b01c-40d1-825b-65cb623e4c5d\") " pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:12.086065 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.086045 2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 16 04:50:12.089548 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.089526 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28zj2\" (UniqueName: \"kubernetes.io/projected/d575250d-2ab4-457d-a702-4386d68d44a3-kube-api-access-28zj2\") pod \"aws-ebs-csi-driver-node-zn4ht\" (UID: \"d575250d-2ab4-457d-a702-4386d68d44a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" Jul 16 04:50:12.089641 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.089528 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf5rm\" (UniqueName: \"kubernetes.io/projected/e18af1c7-b01c-40d1-825b-65cb623e4c5d-kube-api-access-xf5rm\") pod \"multus-additional-cni-plugins-hkq2b\" (UID: \"e18af1c7-b01c-40d1-825b-65cb623e4c5d\") " pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:12.180569 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.180540 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1c5e6cb3-b901-4b3c-a82e-91aa7ccb0ecd-agent-certs\") pod \"konnectivity-agent-9lpb5\" (UID: \"1c5e6cb3-b901-4b3c-a82e-91aa7ccb0ecd\") " pod="kube-system/konnectivity-agent-9lpb5" Jul 16 04:50:12.180753 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.180580 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-host-run-netns\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.180831 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.180758 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-run-systemd\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.180831 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.180783 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn45b\" (UniqueName: \"kubernetes.io/projected/99688117-4206-4fca-be31-0ef08f532483-kube-api-access-bn45b\") pod \"iptables-alerter-n7ljg\" (UID: \"99688117-4206-4fca-be31-0ef08f532483\") " pod="openshift-network-operator/iptables-alerter-n7ljg" Jul 16 04:50:12.180831 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.180800 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-etc-systemd\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.180831 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.180818 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c40a4006-55d3-4359-b554-d71da361a5eb-etc-tuned\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.181025 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.180842 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/607bd4db-30e7-4ddb-9524-42a044c9a203-multus-daemon-config\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.181025 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.180863 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-run-systemd\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.181025 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.180867 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c40a4006-55d3-4359-b554-d71da361a5eb-tmp\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.181025 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.180897 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ffc2facd-a805-43cc-91d2-e35ffff459fd-host\") pod \"node-ca-2lr5q\" (UID: \"ffc2facd-a805-43cc-91d2-e35ffff459fd\") " pod="openshift-image-registry/node-ca-2lr5q" Jul 16 04:50:12.181025 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.180910 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-etc-systemd\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.181025 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.180915 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-host-run-netns\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.181025 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.180963 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ffc2facd-a805-43cc-91d2-e35ffff459fd-host\") pod \"node-ca-2lr5q\" (UID: \"ffc2facd-a805-43cc-91d2-e35ffff459fd\") " pod="openshift-image-registry/node-ca-2lr5q" Jul 16 04:50:12.181025 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.180992 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-cnibin\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.181025 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.180923 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-cnibin\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.181636 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181049 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-run-ovn\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.181636 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181077 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-host-run-ovn-kubernetes\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.181636 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181107 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99688117-4206-4fca-be31-0ef08f532483-iptables-alerter-script\") pod \"iptables-alerter-n7ljg\" (UID: \"99688117-4206-4fca-be31-0ef08f532483\") " pod="openshift-network-operator/iptables-alerter-n7ljg" Jul 16 04:50:12.181636 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181135 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5dtb\" (UniqueName: \"kubernetes.io/projected/ffc2facd-a805-43cc-91d2-e35ffff459fd-kube-api-access-f5dtb\") pod \"node-ca-2lr5q\" (UID: \"ffc2facd-a805-43cc-91d2-e35ffff459fd\") " pod="openshift-image-registry/node-ca-2lr5q" Jul 16 04:50:12.181636 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181160 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-host-run-k8s-cni-cncf-io\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.181636 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181163 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-run-ovn\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.181636 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181185 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-host-var-lib-cni-multus\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.181636 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181224 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-host-var-lib-cni-multus\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.181636 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181228 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/176b1c0f-1174-47ef-aa5c-25e699c63e02-ovn-node-metrics-cert\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.181636 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181259 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqfbb\" (UniqueName: \"kubernetes.io/projected/c40a4006-55d3-4359-b554-d71da361a5eb-kube-api-access-sqfbb\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.181636 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181271 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-host-run-ovn-kubernetes\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.181636 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181284 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-multus-socket-dir-parent\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.181636 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181302 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/176b1c0f-1174-47ef-aa5c-25e699c63e02-env-overrides\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.181636 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181322 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-run\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.181636 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181346 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-multus-conf-dir\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.181636 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181363 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-run-openvswitch\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.181636 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181378 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-log-socket\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.182518 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181402 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-hostroot\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.182518 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181428 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmq5s\" (UniqueName: \"kubernetes.io/projected/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-kube-api-access-kmq5s\") pod \"network-metrics-daemon-2srzm\" (UID: \"8b4ec3ce-70fc-4716-a3c6-5ab123a8be45\") " pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:12.182518 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181471 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/176b1c0f-1174-47ef-aa5c-25e699c63e02-ovnkube-script-lib\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.182518 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181498 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/076bc066-399e-417e-a245-14833a939ca8-hosts-file\") pod \"node-resolver-vkqpj\" (UID: \"076bc066-399e-417e-a245-14833a939ca8\") " pod="openshift-dns/node-resolver-vkqpj" Jul 16 04:50:12.182518 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181519 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs\") pod \"network-metrics-daemon-2srzm\" (UID: \"8b4ec3ce-70fc-4716-a3c6-5ab123a8be45\") " pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:12.182518 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181535 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-host-cni-netd\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.182518 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181550 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-etc-kubernetes\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.182518 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181568 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-host-slash\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.182518 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181602 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-var-lib-openvswitch\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.182518 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181628 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-lib-modules\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.182518 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181673 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq2r5\" (UniqueName: \"kubernetes.io/projected/076bc066-399e-417e-a245-14833a939ca8-kube-api-access-wq2r5\") pod \"node-resolver-vkqpj\" (UID: \"076bc066-399e-417e-a245-14833a939ca8\") " pod="openshift-dns/node-resolver-vkqpj" Jul 16 04:50:12.182518 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181700 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-host-var-lib-cni-bin\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.182518 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181740 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-host-cni-bin\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.182518 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181767 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/176b1c0f-1174-47ef-aa5c-25e699c63e02-ovnkube-config\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.182518 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181776 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/607bd4db-30e7-4ddb-9524-42a044c9a203-multus-daemon-config\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.182518 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181793 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/076bc066-399e-417e-a245-14833a939ca8-tmp-dir\") pod \"node-resolver-vkqpj\" (UID: \"076bc066-399e-417e-a245-14833a939ca8\") " pod="openshift-dns/node-resolver-vkqpj" Jul 16 04:50:12.182518 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181839 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-os-release\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.182518 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181873 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-host-run-multus-certs\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.183496 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181902 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-systemd-units\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.183496 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181936 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-etc-openvswitch\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.183496 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181963 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-etc-sysconfig\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.183496 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.181990 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-multus-cni-dir\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.183496 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.182024 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-host-run-netns\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.183496 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.182075 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-host-var-lib-kubelet\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.183496 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.182083 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99688117-4206-4fca-be31-0ef08f532483-iptables-alerter-script\") pod \"iptables-alerter-n7ljg\" (UID: \"99688117-4206-4fca-be31-0ef08f532483\") " pod="openshift-network-operator/iptables-alerter-n7ljg" Jul 16 04:50:12.183496 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.182104 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngg9s\" (UniqueName: \"kubernetes.io/projected/176b1c0f-1174-47ef-aa5c-25e699c63e02-kube-api-access-ngg9s\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.183496 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.182134 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-etc-sysctl-d\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.183496 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.182159 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-host\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.183496 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.182167 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-host-run-k8s-cni-cncf-io\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.183496 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.182190 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-system-cni-dir\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.183496 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.182218 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm8zs\" (UniqueName: \"kubernetes.io/projected/607bd4db-30e7-4ddb-9524-42a044c9a203-kube-api-access-xm8zs\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.183496 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.182245 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-host-kubelet\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.183496 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.182251 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/076bc066-399e-417e-a245-14833a939ca8-hosts-file\") pod \"node-resolver-vkqpj\" (UID: \"076bc066-399e-417e-a245-14833a939ca8\") " pod="openshift-dns/node-resolver-vkqpj" Jul 16 04:50:12.183496 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.182269 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-etc-modprobe-d\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.183496 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.182379 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-multus-socket-dir-parent\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.183496 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.182414 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-etc-modprobe-d\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.184286 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.182580 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/076bc066-399e-417e-a245-14833a939ca8-tmp-dir\") pod \"node-resolver-vkqpj\" (UID: \"076bc066-399e-417e-a245-14833a939ca8\") " pod="openshift-dns/node-resolver-vkqpj" Jul 16 04:50:12.184286 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.182857 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/176b1c0f-1174-47ef-aa5c-25e699c63e02-env-overrides\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.184286 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.182923 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-host-cni-bin\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.184286 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.182941 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-run\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.184286 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.182992 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-run-openvswitch\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.184286 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.183029 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-multus-conf-dir\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.184286 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.183074 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-log-socket\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.184286 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.183116 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-hostroot\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.184286 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.183148 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-host-kubelet\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.184286 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.183201 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-host-var-lib-kubelet\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.184286 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.183398 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-os-release\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.184286 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.183475 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-host-run-multus-certs\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.184286 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.183516 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-systemd-units\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.184286 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.183552 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-etc-openvswitch\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.184286 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.183596 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-etc-sysconfig\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.184286 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.183645 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-multus-cni-dir\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.184286 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.183713 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/176b1c0f-1174-47ef-aa5c-25e699c63e02-ovnkube-config\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.184286 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.183793 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-host\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.185143 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.183843 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/176b1c0f-1174-47ef-aa5c-25e699c63e02-ovnkube-script-lib\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.185143 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.183888 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-system-cni-dir\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.185143 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.183897 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-var-lib-kubelet\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.185143 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.183913 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-etc-sysctl-d\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.185143 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.183937 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-host-run-netns\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.185143 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.183955 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-var-lib-kubelet\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.185143 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.183977 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-etc-kubernetes\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.185143 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.183975 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-host-cni-netd\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.185143 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.184002 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-host-var-lib-cni-bin\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.185143 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.184008 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-host-slash\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.185143 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.184052 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-var-lib-openvswitch\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.185143 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.184094 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-lib-modules\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.185143 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:12.184125 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:50:12.185143 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.184172 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.185143 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:12.184179 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs podName:8b4ec3ce-70fc-4716-a3c6-5ab123a8be45 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:12.684163745 +0000 UTC m=+3.056286931 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs") pod "network-metrics-daemon-2srzm" (UID: "8b4ec3ce-70fc-4716-a3c6-5ab123a8be45") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:50:12.185143 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.184226 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.185143 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.184267 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-sys\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.185846 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.184297 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c40a4006-55d3-4359-b554-d71da361a5eb-tmp\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.185846 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.184303 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ffc2facd-a805-43cc-91d2-e35ffff459fd-serviceca\") pod \"node-ca-2lr5q\" (UID: \"ffc2facd-a805-43cc-91d2-e35ffff459fd\") " pod="openshift-image-registry/node-ca-2lr5q" Jul 16 04:50:12.185846 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.184342 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-sys\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.185846 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.184388 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/607bd4db-30e7-4ddb-9524-42a044c9a203-cni-binary-copy\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.185846 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.184423 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-etc-kubernetes\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.185846 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.184511 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c40a4006-55d3-4359-b554-d71da361a5eb-etc-tuned\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.185846 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.184536 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1c5e6cb3-b901-4b3c-a82e-91aa7ccb0ecd-konnectivity-ca\") pod \"konnectivity-agent-9lpb5\" (UID: \"1c5e6cb3-b901-4b3c-a82e-91aa7ccb0ecd\") " pod="kube-system/konnectivity-agent-9lpb5" Jul 16 04:50:12.185846 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.184554 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/607bd4db-30e7-4ddb-9524-42a044c9a203-etc-kubernetes\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.185846 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.184709 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs4tn\" (UniqueName: \"kubernetes.io/projected/0f651332-08d0-4eb5-b1e8-5d9437727895-kube-api-access-gs4tn\") pod \"network-check-target-jbmmm\" (UID: \"0f651332-08d0-4eb5-b1e8-5d9437727895\") " pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:12.185846 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.184779 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-node-log\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.185846 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.184809 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99688117-4206-4fca-be31-0ef08f532483-host-slash\") pod \"iptables-alerter-n7ljg\" (UID: \"99688117-4206-4fca-be31-0ef08f532483\") " pod="openshift-network-operator/iptables-alerter-n7ljg" Jul 16 04:50:12.185846 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.184856 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-etc-sysctl-conf\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.185846 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.184956 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1c5e6cb3-b901-4b3c-a82e-91aa7ccb0ecd-agent-certs\") pod \"konnectivity-agent-9lpb5\" (UID: \"1c5e6cb3-b901-4b3c-a82e-91aa7ccb0ecd\") " pod="kube-system/konnectivity-agent-9lpb5" Jul 16 04:50:12.185846 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.184980 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/607bd4db-30e7-4ddb-9524-42a044c9a203-cni-binary-copy\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.185846 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.184997 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ffc2facd-a805-43cc-91d2-e35ffff459fd-serviceca\") pod \"node-ca-2lr5q\" (UID: \"ffc2facd-a805-43cc-91d2-e35ffff459fd\") " pod="openshift-image-registry/node-ca-2lr5q" Jul 16 04:50:12.185846 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.185056 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c40a4006-55d3-4359-b554-d71da361a5eb-etc-sysctl-conf\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.185846 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.185103 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99688117-4206-4fca-be31-0ef08f532483-host-slash\") pod \"iptables-alerter-n7ljg\" (UID: \"99688117-4206-4fca-be31-0ef08f532483\") " pod="openshift-network-operator/iptables-alerter-n7ljg" Jul 16 04:50:12.185846 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.185112 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/176b1c0f-1174-47ef-aa5c-25e699c63e02-node-log\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.186523 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.185228 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1c5e6cb3-b901-4b3c-a82e-91aa7ccb0ecd-konnectivity-ca\") pod \"konnectivity-agent-9lpb5\" (UID: \"1c5e6cb3-b901-4b3c-a82e-91aa7ccb0ecd\") " pod="kube-system/konnectivity-agent-9lpb5" Jul 16 04:50:12.186523 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.185255 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/176b1c0f-1174-47ef-aa5c-25e699c63e02-ovn-node-metrics-cert\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.189874 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.189848 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5dtb\" (UniqueName: \"kubernetes.io/projected/ffc2facd-a805-43cc-91d2-e35ffff459fd-kube-api-access-f5dtb\") pod \"node-ca-2lr5q\" (UID: \"ffc2facd-a805-43cc-91d2-e35ffff459fd\") " pod="openshift-image-registry/node-ca-2lr5q" Jul 16 04:50:12.190078 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.190052 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn45b\" (UniqueName: \"kubernetes.io/projected/99688117-4206-4fca-be31-0ef08f532483-kube-api-access-bn45b\") pod \"iptables-alerter-n7ljg\" (UID: \"99688117-4206-4fca-be31-0ef08f532483\") " pod="openshift-network-operator/iptables-alerter-n7ljg" Jul 16 04:50:12.190622 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.190594 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqfbb\" (UniqueName: \"kubernetes.io/projected/c40a4006-55d3-4359-b554-d71da361a5eb-kube-api-access-sqfbb\") pod \"tuned-ww44m\" (UID: \"c40a4006-55d3-4359-b554-d71da361a5eb\") " pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.190779 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:12.190737 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 04:50:12.190853 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:12.190785 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 04:50:12.190853 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:12.190799 2561 projected.go:196] Error preparing data for projected volume kube-api-access-gs4tn for pod openshift-network-diagnostics/network-check-target-jbmmm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:50:12.190978 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:12.190887 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f651332-08d0-4eb5-b1e8-5d9437727895-kube-api-access-gs4tn podName:0f651332-08d0-4eb5-b1e8-5d9437727895 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:12.690864264 +0000 UTC m=+3.062987478 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gs4tn" (UniqueName: "kubernetes.io/projected/0f651332-08d0-4eb5-b1e8-5d9437727895-kube-api-access-gs4tn") pod "network-check-target-jbmmm" (UID: "0f651332-08d0-4eb5-b1e8-5d9437727895") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:50:12.192329 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.192306 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm8zs\" (UniqueName: \"kubernetes.io/projected/607bd4db-30e7-4ddb-9524-42a044c9a203-kube-api-access-xm8zs\") pod \"multus-phkqm\" (UID: \"607bd4db-30e7-4ddb-9524-42a044c9a203\") " pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.192948 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.192926 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq2r5\" (UniqueName: \"kubernetes.io/projected/076bc066-399e-417e-a245-14833a939ca8-kube-api-access-wq2r5\") pod \"node-resolver-vkqpj\" (UID: \"076bc066-399e-417e-a245-14833a939ca8\") " pod="openshift-dns/node-resolver-vkqpj" Jul 16 04:50:12.193016 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.193008 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmq5s\" (UniqueName: \"kubernetes.io/projected/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-kube-api-access-kmq5s\") pod \"network-metrics-daemon-2srzm\" (UID: \"8b4ec3ce-70fc-4716-a3c6-5ab123a8be45\") " pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:12.194696 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.194672 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngg9s\" (UniqueName: \"kubernetes.io/projected/176b1c0f-1174-47ef-aa5c-25e699c63e02-kube-api-access-ngg9s\") pod \"ovnkube-node-bsqdp\" (UID: \"176b1c0f-1174-47ef-aa5c-25e699c63e02\") " pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.272619 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.272522 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hkq2b" Jul 16 04:50:12.281543 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.281515 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" Jul 16 04:50:12.291578 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.291552 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ww44m" Jul 16 04:50:12.298351 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.298325 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2lr5q" Jul 16 04:50:12.305987 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.305952 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vkqpj" Jul 16 04:50:12.314957 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.314927 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:12.322760 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.322737 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-n7ljg" Jul 16 04:50:12.331782 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.331760 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-phkqm" Jul 16 04:50:12.336744 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.336730 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9lpb5" Jul 16 04:50:12.586935 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.586899 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-original-pull-secret\") pod \"global-pull-secret-syncer-p97p6\" (UID: \"67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c\") " pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:12.587136 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:12.587074 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 04:50:12.587202 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:12.587150 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-original-pull-secret podName:67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c nodeName:}" failed. No retries permitted until 2026-07-16 04:50:13.587128658 +0000 UTC m=+3.959251858 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-original-pull-secret") pod "global-pull-secret-syncer-p97p6" (UID: "67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c") : object "kube-system"/"original-pull-secret" not registered Jul 16 04:50:12.688103 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.688070 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs\") pod \"network-metrics-daemon-2srzm\" (UID: \"8b4ec3ce-70fc-4716-a3c6-5ab123a8be45\") " pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:12.688260 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:12.688225 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:50:12.688322 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:12.688293 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs podName:8b4ec3ce-70fc-4716-a3c6-5ab123a8be45 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:13.688275996 +0000 UTC m=+4.060399181 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs") pod "network-metrics-daemon-2srzm" (UID: "8b4ec3ce-70fc-4716-a3c6-5ab123a8be45") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:50:12.789286 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:12.789252 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs4tn\" (UniqueName: \"kubernetes.io/projected/0f651332-08d0-4eb5-b1e8-5d9437727895-kube-api-access-gs4tn\") pod \"network-check-target-jbmmm\" (UID: \"0f651332-08d0-4eb5-b1e8-5d9437727895\") " pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:12.789474 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:12.789441 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 04:50:12.789474 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:12.789467 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 04:50:12.789595 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:12.789481 2561 projected.go:196] Error preparing data for projected volume kube-api-access-gs4tn for pod openshift-network-diagnostics/network-check-target-jbmmm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:50:12.789595 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:12.789551 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f651332-08d0-4eb5-b1e8-5d9437727895-kube-api-access-gs4tn podName:0f651332-08d0-4eb5-b1e8-5d9437727895 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:13.789531007 +0000 UTC m=+4.161654210 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gs4tn" (UniqueName: "kubernetes.io/projected/0f651332-08d0-4eb5-b1e8-5d9437727895-kube-api-access-gs4tn") pod "network-check-target-jbmmm" (UID: "0f651332-08d0-4eb5-b1e8-5d9437727895") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:50:13.011950 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:13.011869 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 04:45:12 +0000 UTC" deadline="2027-12-13 05:40:04.341721642 +0000 UTC" Jul 16 04:50:13.011950 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:13.011930 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12360h49m51.329796099s" Jul 16 04:50:13.095062 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:13.095031 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:13.095062 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:13.095058 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:13.095228 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:13.095154 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:13.095228 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:13.095162 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p97p6" podUID="67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c" Jul 16 04:50:13.095336 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:13.095255 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2srzm" podUID="8b4ec3ce-70fc-4716-a3c6-5ab123a8be45" Jul 16 04:50:13.095336 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:13.095292 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbmmm" podUID="0f651332-08d0-4eb5-b1e8-5d9437727895" Jul 16 04:50:13.098721 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:13.098499 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod076bc066_399e_417e_a245_14833a939ca8.slice/crio-f83c76efae3c8a32f04791fdba6dfdc04fb6596b0426ea3a8406654cef087a38 WatchSource:0}: Error finding container f83c76efae3c8a32f04791fdba6dfdc04fb6596b0426ea3a8406654cef087a38: Status 404 returned error can't find the container with id f83c76efae3c8a32f04791fdba6dfdc04fb6596b0426ea3a8406654cef087a38 Jul 16 04:50:13.103125 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:13.103102 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vkqpj" event={"ID":"076bc066-399e-417e-a245-14833a939ca8","Type":"ContainerStarted","Data":"f83c76efae3c8a32f04791fdba6dfdc04fb6596b0426ea3a8406654cef087a38"} Jul 16 04:50:13.103366 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:13.103345 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod607bd4db_30e7_4ddb_9524_42a044c9a203.slice/crio-e87ed81bc04dbfef0f400f69515349e77089532ce8fcd8dd7804381b3e6839e7 WatchSource:0}: Error finding container e87ed81bc04dbfef0f400f69515349e77089532ce8fcd8dd7804381b3e6839e7: Status 404 returned error can't find the container with id e87ed81bc04dbfef0f400f69515349e77089532ce8fcd8dd7804381b3e6839e7 Jul 16 04:50:13.104611 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:13.104587 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffc2facd_a805_43cc_91d2_e35ffff459fd.slice/crio-deee5a46ef736ee9bfad8f8db2e48a2edd3df67099c3a92a23618dbe03999d3b WatchSource:0}: Error finding container deee5a46ef736ee9bfad8f8db2e48a2edd3df67099c3a92a23618dbe03999d3b: Status 404 returned error can't find the container with id deee5a46ef736ee9bfad8f8db2e48a2edd3df67099c3a92a23618dbe03999d3b Jul 16 04:50:13.105433 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:13.105397 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd575250d_2ab4_457d_a702_4386d68d44a3.slice/crio-212829ae857f2a91584aa7a33545cc77b11d9ebbe83965e527045f78766b03fe WatchSource:0}: Error finding container 212829ae857f2a91584aa7a33545cc77b11d9ebbe83965e527045f78766b03fe: Status 404 returned error can't find the container with id 212829ae857f2a91584aa7a33545cc77b11d9ebbe83965e527045f78766b03fe Jul 16 04:50:13.108197 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:13.108173 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99688117_4206_4fca_be31_0ef08f532483.slice/crio-0323aa579587c69ac18a813adc1c6e416b75b8c2f88f9a6f7d10b1d2f7777533 WatchSource:0}: Error finding container 0323aa579587c69ac18a813adc1c6e416b75b8c2f88f9a6f7d10b1d2f7777533: Status 404 returned error can't find the container with id 0323aa579587c69ac18a813adc1c6e416b75b8c2f88f9a6f7d10b1d2f7777533 Jul 16 04:50:13.108935 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:13.108434 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode18af1c7_b01c_40d1_825b_65cb623e4c5d.slice/crio-0a89a21cfaf89e052d9662218d1768f60ba207d6fe3837d38411b22e09456ac0 WatchSource:0}: Error finding container 0a89a21cfaf89e052d9662218d1768f60ba207d6fe3837d38411b22e09456ac0: Status 404 returned error can't find the container with id 0a89a21cfaf89e052d9662218d1768f60ba207d6fe3837d38411b22e09456ac0 Jul 16 04:50:13.109335 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:13.109299 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod176b1c0f_1174_47ef_aa5c_25e699c63e02.slice/crio-478f9fb1a26c51e9b46660bfb983bc355c833714d564c6cf32f3bf378c49c07f WatchSource:0}: Error finding container 478f9fb1a26c51e9b46660bfb983bc355c833714d564c6cf32f3bf378c49c07f: Status 404 returned error can't find the container with id 478f9fb1a26c51e9b46660bfb983bc355c833714d564c6cf32f3bf378c49c07f Jul 16 04:50:13.111133 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:13.111113 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc40a4006_55d3_4359_b554_d71da361a5eb.slice/crio-17315660671e019226160df26cf85276ecdf4a787a7f46b81261eb677bef9a66 WatchSource:0}: Error finding container 17315660671e019226160df26cf85276ecdf4a787a7f46b81261eb677bef9a66: Status 404 returned error can't find the container with id 17315660671e019226160df26cf85276ecdf4a787a7f46b81261eb677bef9a66 Jul 16 04:50:13.113742 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:13.113704 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c5e6cb3_b901_4b3c_a82e_91aa7ccb0ecd.slice/crio-94baad5a122ada483df9c78102eaefdc8d93552766fe1d388e3397e7e00ebdc0 WatchSource:0}: Error finding container 94baad5a122ada483df9c78102eaefdc8d93552766fe1d388e3397e7e00ebdc0: Status 404 returned error can't find the container with id 94baad5a122ada483df9c78102eaefdc8d93552766fe1d388e3397e7e00ebdc0 Jul 16 04:50:13.596826 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:13.596729 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-original-pull-secret\") pod \"global-pull-secret-syncer-p97p6\" (UID: \"67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c\") " pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:13.596996 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:13.596925 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 04:50:13.597054 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:13.596996 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-original-pull-secret podName:67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c nodeName:}" failed. No retries permitted until 2026-07-16 04:50:15.596976862 +0000 UTC m=+5.969100049 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-original-pull-secret") pod "global-pull-secret-syncer-p97p6" (UID: "67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c") : object "kube-system"/"original-pull-secret" not registered Jul 16 04:50:13.612298 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:13.612267 2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 04:50:13.697424 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:13.697387 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs\") pod \"network-metrics-daemon-2srzm\" (UID: \"8b4ec3ce-70fc-4716-a3c6-5ab123a8be45\") " pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:13.697595 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:13.697574 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:50:13.697674 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:13.697652 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs podName:8b4ec3ce-70fc-4716-a3c6-5ab123a8be45 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:15.697631366 +0000 UTC m=+6.069754551 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs") pod "network-metrics-daemon-2srzm" (UID: "8b4ec3ce-70fc-4716-a3c6-5ab123a8be45") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:50:13.798639 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:13.798597 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs4tn\" (UniqueName: \"kubernetes.io/projected/0f651332-08d0-4eb5-b1e8-5d9437727895-kube-api-access-gs4tn\") pod \"network-check-target-jbmmm\" (UID: \"0f651332-08d0-4eb5-b1e8-5d9437727895\") " pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:13.798889 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:13.798866 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 04:50:13.798973 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:13.798899 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 04:50:13.798973 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:13.798917 2561 projected.go:196] Error preparing data for projected volume kube-api-access-gs4tn for pod openshift-network-diagnostics/network-check-target-jbmmm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:50:13.799082 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:13.798981 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f651332-08d0-4eb5-b1e8-5d9437727895-kube-api-access-gs4tn podName:0f651332-08d0-4eb5-b1e8-5d9437727895 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:15.798960553 +0000 UTC m=+6.171083753 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gs4tn" (UniqueName: "kubernetes.io/projected/0f651332-08d0-4eb5-b1e8-5d9437727895-kube-api-access-gs4tn") pod "network-check-target-jbmmm" (UID: "0f651332-08d0-4eb5-b1e8-5d9437727895") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:50:14.012990 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:14.012907 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 04:45:12 +0000 UTC" deadline="2028-02-04 13:21:10.718663029 +0000 UTC" Jul 16 04:50:14.012990 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:14.012946 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13640h30m56.705722278s" Jul 16 04:50:14.109469 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:14.109431 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9lpb5" event={"ID":"1c5e6cb3-b901-4b3c-a82e-91aa7ccb0ecd","Type":"ContainerStarted","Data":"94baad5a122ada483df9c78102eaefdc8d93552766fe1d388e3397e7e00ebdc0"} Jul 16 04:50:14.120889 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:14.120784 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ww44m" event={"ID":"c40a4006-55d3-4359-b554-d71da361a5eb","Type":"ContainerStarted","Data":"17315660671e019226160df26cf85276ecdf4a787a7f46b81261eb677bef9a66"} Jul 16 04:50:14.123532 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:14.123414 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" event={"ID":"176b1c0f-1174-47ef-aa5c-25e699c63e02","Type":"ContainerStarted","Data":"478f9fb1a26c51e9b46660bfb983bc355c833714d564c6cf32f3bf378c49c07f"} Jul 16 04:50:14.126413 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:14.126324 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" event={"ID":"d575250d-2ab4-457d-a702-4386d68d44a3","Type":"ContainerStarted","Data":"212829ae857f2a91584aa7a33545cc77b11d9ebbe83965e527045f78766b03fe"} Jul 16 04:50:14.129897 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:14.129233 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-187.ec2.internal" event={"ID":"d36fd6545199dd402a0bbf93c6fc2c71","Type":"ContainerStarted","Data":"eb06f9e9fe3d87ee562656647c43081cca29aec616b9bb705a550b8bd3b938e5"} Jul 16 04:50:14.134185 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:14.134114 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hkq2b" event={"ID":"e18af1c7-b01c-40d1-825b-65cb623e4c5d","Type":"ContainerStarted","Data":"0a89a21cfaf89e052d9662218d1768f60ba207d6fe3837d38411b22e09456ac0"} Jul 16 04:50:14.137941 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:14.137918 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-n7ljg" event={"ID":"99688117-4206-4fca-be31-0ef08f532483","Type":"ContainerStarted","Data":"0323aa579587c69ac18a813adc1c6e416b75b8c2f88f9a6f7d10b1d2f7777533"} Jul 16 04:50:14.139610 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:14.139565 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2lr5q" event={"ID":"ffc2facd-a805-43cc-91d2-e35ffff459fd","Type":"ContainerStarted","Data":"deee5a46ef736ee9bfad8f8db2e48a2edd3df67099c3a92a23618dbe03999d3b"} Jul 16 04:50:14.142239 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:14.142171 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-phkqm" event={"ID":"607bd4db-30e7-4ddb-9524-42a044c9a203","Type":"ContainerStarted","Data":"e87ed81bc04dbfef0f400f69515349e77089532ce8fcd8dd7804381b3e6839e7"} Jul 16 04:50:14.150094 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:14.149542 2561 generic.go:356] "Generic (PLEG): container finished" podID="7dae16fba94631b5028f0bd95f98f48b" containerID="5a18bce16dbcb3021585b070387bff32714de02e81976b890b6410a961653369" exitCode=0 Jul 16 04:50:14.150094 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:14.149590 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-187.ec2.internal" event={"ID":"7dae16fba94631b5028f0bd95f98f48b","Type":"ContainerDied","Data":"5a18bce16dbcb3021585b070387bff32714de02e81976b890b6410a961653369"} Jul 16 04:50:14.163241 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:14.163196 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-187.ec2.internal" podStartSLOduration=3.163180565 podStartE2EDuration="3.163180565s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:50:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:50:14.144025496 +0000 UTC m=+4.516148700" watchObservedRunningTime="2026-07-16 04:50:14.163180565 +0000 UTC m=+4.535303771" Jul 16 04:50:15.095549 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:15.095018 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:15.095549 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:15.095055 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:15.095549 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:15.095018 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:15.095549 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:15.095147 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p97p6" podUID="67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c" Jul 16 04:50:15.095549 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:15.095218 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2srzm" podUID="8b4ec3ce-70fc-4716-a3c6-5ab123a8be45" Jul 16 04:50:15.095549 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:15.095280 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbmmm" podUID="0f651332-08d0-4eb5-b1e8-5d9437727895" Jul 16 04:50:15.179832 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:15.179800 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-187.ec2.internal" event={"ID":"7dae16fba94631b5028f0bd95f98f48b","Type":"ContainerStarted","Data":"c972e31536393639b686e4b657886e69f2beab4ccdcb2065cf9e3db2b5d8ac08"} Jul 16 04:50:15.611989 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:15.611953 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-original-pull-secret\") pod \"global-pull-secret-syncer-p97p6\" (UID: \"67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c\") " pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:15.612164 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:15.612105 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 04:50:15.612237 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:15.612167 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-original-pull-secret podName:67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c nodeName:}" failed. No retries permitted until 2026-07-16 04:50:19.61214921 +0000 UTC m=+9.984272394 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-original-pull-secret") pod "global-pull-secret-syncer-p97p6" (UID: "67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c") : object "kube-system"/"original-pull-secret" not registered Jul 16 04:50:15.713876 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:15.713842 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs\") pod \"network-metrics-daemon-2srzm\" (UID: \"8b4ec3ce-70fc-4716-a3c6-5ab123a8be45\") " pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:15.714035 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:15.713962 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:50:15.714035 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:15.714009 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs podName:8b4ec3ce-70fc-4716-a3c6-5ab123a8be45 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:19.713991864 +0000 UTC m=+10.086115051 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs") pod "network-metrics-daemon-2srzm" (UID: "8b4ec3ce-70fc-4716-a3c6-5ab123a8be45") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:50:15.815062 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:15.815028 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs4tn\" (UniqueName: \"kubernetes.io/projected/0f651332-08d0-4eb5-b1e8-5d9437727895-kube-api-access-gs4tn\") pod \"network-check-target-jbmmm\" (UID: \"0f651332-08d0-4eb5-b1e8-5d9437727895\") " pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:15.815267 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:15.815225 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 04:50:15.815267 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:15.815246 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 04:50:15.815267 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:15.815261 2561 projected.go:196] Error preparing data for projected volume kube-api-access-gs4tn for pod openshift-network-diagnostics/network-check-target-jbmmm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:50:15.815433 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:15.815318 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f651332-08d0-4eb5-b1e8-5d9437727895-kube-api-access-gs4tn podName:0f651332-08d0-4eb5-b1e8-5d9437727895 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:19.815299307 +0000 UTC m=+10.187422492 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gs4tn" (UniqueName: "kubernetes.io/projected/0f651332-08d0-4eb5-b1e8-5d9437727895-kube-api-access-gs4tn") pod "network-check-target-jbmmm" (UID: "0f651332-08d0-4eb5-b1e8-5d9437727895") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:50:17.095355 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:17.095319 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:17.095825 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:17.095446 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p97p6" podUID="67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c" Jul 16 04:50:17.095888 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:17.095845 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:17.095953 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:17.095936 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbmmm" podUID="0f651332-08d0-4eb5-b1e8-5d9437727895" Jul 16 04:50:17.096009 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:17.095988 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:17.096104 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:17.096075 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2srzm" podUID="8b4ec3ce-70fc-4716-a3c6-5ab123a8be45" Jul 16 04:50:19.095305 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:19.095275 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:19.095803 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:19.095389 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p97p6" podUID="67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c" Jul 16 04:50:19.095803 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:19.095755 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:19.095938 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:19.095857 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbmmm" podUID="0f651332-08d0-4eb5-b1e8-5d9437727895" Jul 16 04:50:19.095938 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:19.095926 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:19.096018 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:19.096002 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2srzm" podUID="8b4ec3ce-70fc-4716-a3c6-5ab123a8be45" Jul 16 04:50:19.645745 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:19.645705 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-original-pull-secret\") pod \"global-pull-secret-syncer-p97p6\" (UID: \"67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c\") " pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:19.645902 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:19.645853 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 04:50:19.645902 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:19.645900 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-original-pull-secret podName:67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c nodeName:}" failed. No retries permitted until 2026-07-16 04:50:27.645888358 +0000 UTC m=+18.018011541 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-original-pull-secret") pod "global-pull-secret-syncer-p97p6" (UID: "67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c") : object "kube-system"/"original-pull-secret" not registered Jul 16 04:50:19.746224 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:19.746186 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs\") pod \"network-metrics-daemon-2srzm\" (UID: \"8b4ec3ce-70fc-4716-a3c6-5ab123a8be45\") " pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:19.746396 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:19.746358 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:50:19.746467 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:19.746416 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs podName:8b4ec3ce-70fc-4716-a3c6-5ab123a8be45 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:27.746401004 +0000 UTC m=+18.118524188 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs") pod "network-metrics-daemon-2srzm" (UID: "8b4ec3ce-70fc-4716-a3c6-5ab123a8be45") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:50:19.847151 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:19.847113 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs4tn\" (UniqueName: \"kubernetes.io/projected/0f651332-08d0-4eb5-b1e8-5d9437727895-kube-api-access-gs4tn\") pod \"network-check-target-jbmmm\" (UID: \"0f651332-08d0-4eb5-b1e8-5d9437727895\") " pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:19.847324 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:19.847312 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 04:50:19.847394 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:19.847337 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 04:50:19.847394 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:19.847352 2561 projected.go:196] Error preparing data for projected volume kube-api-access-gs4tn for pod openshift-network-diagnostics/network-check-target-jbmmm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:50:19.847502 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:19.847413 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f651332-08d0-4eb5-b1e8-5d9437727895-kube-api-access-gs4tn podName:0f651332-08d0-4eb5-b1e8-5d9437727895 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:27.847394188 +0000 UTC m=+18.219517396 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gs4tn" (UniqueName: "kubernetes.io/projected/0f651332-08d0-4eb5-b1e8-5d9437727895-kube-api-access-gs4tn") pod "network-check-target-jbmmm" (UID: "0f651332-08d0-4eb5-b1e8-5d9437727895") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:50:21.094718 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:21.094668 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:21.095232 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:21.094822 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2srzm" podUID="8b4ec3ce-70fc-4716-a3c6-5ab123a8be45" Jul 16 04:50:21.095232 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:21.095182 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:21.095314 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:21.095254 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p97p6" podUID="67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c" Jul 16 04:50:21.095314 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:21.095289 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:21.095378 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:21.095330 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbmmm" podUID="0f651332-08d0-4eb5-b1e8-5d9437727895" Jul 16 04:50:23.095521 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:23.095481 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:23.096013 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:23.095610 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbmmm" podUID="0f651332-08d0-4eb5-b1e8-5d9437727895" Jul 16 04:50:23.096013 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:23.095500 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:23.096013 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:23.095642 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:23.096013 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:23.095741 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p97p6" podUID="67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c" Jul 16 04:50:23.096013 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:23.095831 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2srzm" podUID="8b4ec3ce-70fc-4716-a3c6-5ab123a8be45" Jul 16 04:50:25.095396 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:25.095368 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:25.095396 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:25.095391 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:25.095863 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:25.095367 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:25.095863 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:25.095479 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p97p6" podUID="67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c" Jul 16 04:50:25.095863 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:25.095591 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbmmm" podUID="0f651332-08d0-4eb5-b1e8-5d9437727895" Jul 16 04:50:25.095863 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:25.095676 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2srzm" podUID="8b4ec3ce-70fc-4716-a3c6-5ab123a8be45" Jul 16 04:50:27.094888 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:27.094847 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:27.095330 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:27.094949 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbmmm" podUID="0f651332-08d0-4eb5-b1e8-5d9437727895" Jul 16 04:50:27.095330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:27.094983 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:27.095330 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:27.094863 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:27.095330 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:27.095083 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2srzm" podUID="8b4ec3ce-70fc-4716-a3c6-5ab123a8be45" Jul 16 04:50:27.095330 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:27.095139 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p97p6" podUID="67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c" Jul 16 04:50:27.703419 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:27.703336 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-original-pull-secret\") pod \"global-pull-secret-syncer-p97p6\" (UID: \"67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c\") " pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:27.703570 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:27.703494 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 04:50:27.703570 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:27.703568 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-original-pull-secret podName:67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c nodeName:}" failed. No retries permitted until 2026-07-16 04:50:43.703548552 +0000 UTC m=+34.075671748 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-original-pull-secret") pod "global-pull-secret-syncer-p97p6" (UID: "67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c") : object "kube-system"/"original-pull-secret" not registered Jul 16 04:50:27.804080 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:27.804047 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs\") pod \"network-metrics-daemon-2srzm\" (UID: \"8b4ec3ce-70fc-4716-a3c6-5ab123a8be45\") " pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:27.804259 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:27.804204 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:50:27.804304 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:27.804269 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs podName:8b4ec3ce-70fc-4716-a3c6-5ab123a8be45 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:43.804247284 +0000 UTC m=+34.176370487 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs") pod "network-metrics-daemon-2srzm" (UID: "8b4ec3ce-70fc-4716-a3c6-5ab123a8be45") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:50:27.904933 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:27.904900 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs4tn\" (UniqueName: \"kubernetes.io/projected/0f651332-08d0-4eb5-b1e8-5d9437727895-kube-api-access-gs4tn\") pod \"network-check-target-jbmmm\" (UID: \"0f651332-08d0-4eb5-b1e8-5d9437727895\") " pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:27.905107 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:27.905040 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 04:50:27.905107 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:27.905099 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 04:50:27.905180 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:27.905113 2561 projected.go:196] Error preparing data for projected volume kube-api-access-gs4tn for pod openshift-network-diagnostics/network-check-target-jbmmm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:50:27.905180 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:27.905162 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f651332-08d0-4eb5-b1e8-5d9437727895-kube-api-access-gs4tn podName:0f651332-08d0-4eb5-b1e8-5d9437727895 nodeName:}" failed. No retries permitted until 2026-07-16 04:50:43.905148366 +0000 UTC m=+34.277271550 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gs4tn" (UniqueName: "kubernetes.io/projected/0f651332-08d0-4eb5-b1e8-5d9437727895-kube-api-access-gs4tn") pod "network-check-target-jbmmm" (UID: "0f651332-08d0-4eb5-b1e8-5d9437727895") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:50:29.095506 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:29.095421 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:29.095506 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:29.095464 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:29.096036 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:29.095527 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:29.096036 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:29.095547 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p97p6" podUID="67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c" Jul 16 04:50:29.096036 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:29.095629 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2srzm" podUID="8b4ec3ce-70fc-4716-a3c6-5ab123a8be45" Jul 16 04:50:29.096036 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:29.095753 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbmmm" podUID="0f651332-08d0-4eb5-b1e8-5d9437727895" Jul 16 04:50:31.094940 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:31.094903 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:31.095394 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:31.095028 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:31.095394 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:31.095038 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:31.095394 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:31.095036 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p97p6" podUID="67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c" Jul 16 04:50:31.095394 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:31.095146 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbmmm" podUID="0f651332-08d0-4eb5-b1e8-5d9437727895" Jul 16 04:50:31.095394 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:31.095271 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2srzm" podUID="8b4ec3ce-70fc-4716-a3c6-5ab123a8be45" Jul 16 04:50:32.220658 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:32.220253 2561 generic.go:356] "Generic (PLEG): container finished" podID="e18af1c7-b01c-40d1-825b-65cb623e4c5d" containerID="3e9c6964ba21cce680ea9545839c1107171dcb282599d0c1c651f73d8549f5a6" exitCode=0 Jul 16 04:50:32.221458 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:32.220334 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hkq2b" event={"ID":"e18af1c7-b01c-40d1-825b-65cb623e4c5d","Type":"ContainerDied","Data":"3e9c6964ba21cce680ea9545839c1107171dcb282599d0c1c651f73d8549f5a6"} Jul 16 04:50:32.222347 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:32.222322 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2lr5q" event={"ID":"ffc2facd-a805-43cc-91d2-e35ffff459fd","Type":"ContainerStarted","Data":"686e9c1d2797d6140c63e7efa2f074f00c336b3526ace20bef397be6544eccab"} Jul 16 04:50:32.223737 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:32.223714 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-phkqm" event={"ID":"607bd4db-30e7-4ddb-9524-42a044c9a203","Type":"ContainerStarted","Data":"6ce0f957bb8fa7907c650c7858f7c4039c32e734cce46c4f4d01b423d1bd1c56"} Jul 16 04:50:32.225156 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:32.225129 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vkqpj" event={"ID":"076bc066-399e-417e-a245-14833a939ca8","Type":"ContainerStarted","Data":"7d52589c2dfad48229a8ff06013fc6272b6a2f9d491224fe18f3386f636ff562"} Jul 16 04:50:32.226537 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:32.226515 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9lpb5" event={"ID":"1c5e6cb3-b901-4b3c-a82e-91aa7ccb0ecd","Type":"ContainerStarted","Data":"bf028f8740d5590f08a023a5b5939e022a5765045af4693f422093247e450f2b"} Jul 16 04:50:32.227840 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:32.227818 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ww44m" event={"ID":"c40a4006-55d3-4359-b554-d71da361a5eb","Type":"ContainerStarted","Data":"257b7db125208a1f14815328ec5a473c4426a376154c932885c32d7eb5ab65a8"} Jul 16 04:50:32.229865 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:32.229847 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" event={"ID":"176b1c0f-1174-47ef-aa5c-25e699c63e02","Type":"ContainerStarted","Data":"cd4c38f133da9b6fdb1f40a51b4b86fc86b81e0f4f24d582f18bd532ebddc75f"} Jul 16 04:50:32.229963 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:32.229904 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" event={"ID":"176b1c0f-1174-47ef-aa5c-25e699c63e02","Type":"ContainerStarted","Data":"cfde49a6fcf57cb8f9ea7026fff2d3b21c10d64d701b7695a17b30c298eb0021"} Jul 16 04:50:32.229963 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:32.229934 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" event={"ID":"176b1c0f-1174-47ef-aa5c-25e699c63e02","Type":"ContainerStarted","Data":"569acfa2d5028226f7c0b1306a5bf4d5cc1e969a8005642c303adaa985b7b2c0"} Jul 16 04:50:32.231251 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:32.231227 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" event={"ID":"d575250d-2ab4-457d-a702-4386d68d44a3","Type":"ContainerStarted","Data":"08b2d49695196c8bbbc4e9aaef7e98ed6eaa09a6038b8fcf9409241bf721eb7a"} Jul 16 04:50:32.249953 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:32.249912 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-187.ec2.internal" podStartSLOduration=21.249899323 podStartE2EDuration="21.249899323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:50:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:50:15.194279575 +0000 UTC m=+5.566402780" watchObservedRunningTime="2026-07-16 04:50:32.249899323 +0000 UTC m=+22.622022529" Jul 16 04:50:32.259963 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:32.259925 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9lpb5" podStartSLOduration=13.57229541 podStartE2EDuration="22.259911661s" totalImagesPullingTime="8.687616251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:50:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:50:32.259431255 +0000 UTC m=+22.631554481" watchObservedRunningTime="2026-07-16 04:50:32.259911661 +0000 UTC m=+22.632034866" Jul 16 04:50:32.271942 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:32.271907 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2lr5q" podStartSLOduration=7.969786614 podStartE2EDuration="22.271893919s" totalImagesPullingTime="14.302107305s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:50:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:50:32.271574822 +0000 UTC m=+22.643698028" watchObservedRunningTime="2026-07-16 04:50:32.271893919 +0000 UTC m=+22.644017124" Jul 16 04:50:32.285353 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:32.285316 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vkqpj" podStartSLOduration=3.770474082 podStartE2EDuration="22.285304239s" totalImagesPullingTime="18.514830157s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:50:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:50:32.285094919 +0000 UTC m=+22.657218124" watchObservedRunningTime="2026-07-16 04:50:32.285304239 +0000 UTC m=+22.657427444" Jul 16 04:50:32.300622 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:32.300589 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ww44m" podStartSLOduration=3.798274116 podStartE2EDuration="22.300579081s" totalImagesPullingTime="18.502304965s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:50:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:50:32.300366196 +0000 UTC m=+22.672489400" watchObservedRunningTime="2026-07-16 04:50:32.300579081 +0000 UTC m=+22.672702286" Jul 16 04:50:32.317401 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:32.317369 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-phkqm" podStartSLOduration=3.714390716 podStartE2EDuration="22.317358098s" totalImagesPullingTime="18.602967382s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:50:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:50:32.317224511 +0000 UTC m=+22.689347715" watchObservedRunningTime="2026-07-16 04:50:32.317358098 +0000 UTC m=+22.689481303" Jul 16 04:50:32.887067 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:32.887037 2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 16 04:50:33.033589 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:33.033448 2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T04:50:32.887058264Z","UUID":"e728e8d3-eb4b-4eb7-9bcc-1f25f8e8442f","Handler":null,"Name":"","Endpoint":""} Jul 16 04:50:33.035107 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:33.035087 2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 16 04:50:33.035224 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:33.035142 2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 16 04:50:33.094643 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:33.094610 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:33.094824 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:33.094654 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:33.094824 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:33.094610 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:33.094824 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:33.094755 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p97p6" podUID="67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c" Jul 16 04:50:33.095022 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:33.095003 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbmmm" podUID="0f651332-08d0-4eb5-b1e8-5d9437727895" Jul 16 04:50:33.095115 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:33.095098 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2srzm" podUID="8b4ec3ce-70fc-4716-a3c6-5ab123a8be45" Jul 16 04:50:33.236798 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:33.236764 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" event={"ID":"176b1c0f-1174-47ef-aa5c-25e699c63e02","Type":"ContainerStarted","Data":"400dd5852ee8f76f22f127704278098dc0e8ca2e00538ceabe79dbbdf6bf99b8"} Jul 16 04:50:33.237254 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:33.236849 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" event={"ID":"176b1c0f-1174-47ef-aa5c-25e699c63e02","Type":"ContainerStarted","Data":"b72bffdcb9ab1eb88bf6e0850953787b9e6eb8d412bb2fe5bcc2d8968b3a68f5"} Jul 16 04:50:33.237254 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:33.236901 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" event={"ID":"176b1c0f-1174-47ef-aa5c-25e699c63e02","Type":"ContainerStarted","Data":"cd7ba875e1074413cc8e04538d53ea4acdb7bd94ad463d2166c93ed5a7c084b1"} Jul 16 04:50:33.238746 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:33.238717 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" event={"ID":"d575250d-2ab4-457d-a702-4386d68d44a3","Type":"ContainerStarted","Data":"50b667993fe573ea929be79a852298f6b204f94562e85af6b0163152611a5224"} Jul 16 04:50:33.240126 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:33.240062 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-n7ljg" event={"ID":"99688117-4206-4fca-be31-0ef08f532483","Type":"ContainerStarted","Data":"b06b245b65aa5fac102165e92035d8119a50bebc9c385e30452a9e11f5937f31"} Jul 16 04:50:33.255031 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:33.254985 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-n7ljg" podStartSLOduration=4.748542015 podStartE2EDuration="23.254973021s" totalImagesPullingTime="18.506431006s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:50:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:50:33.254638314 +0000 UTC m=+23.626761520" watchObservedRunningTime="2026-07-16 04:50:33.254973021 +0000 UTC m=+23.627096226" Jul 16 04:50:34.245156 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:34.244947 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" event={"ID":"d575250d-2ab4-457d-a702-4386d68d44a3","Type":"ContainerStarted","Data":"6f030a1dca7115f03c8a91b0d76782fa3a8ac4fff245e0fbbb61e045ce895c2e"} Jul 16 04:50:34.262542 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:34.262458 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zn4ht" podStartSLOduration=3.667873048 podStartE2EDuration="24.262441091s" totalImagesPullingTime="20.594568043s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:50:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:50:34.262363323 +0000 UTC m=+24.634486523" watchObservedRunningTime="2026-07-16 04:50:34.262441091 +0000 UTC m=+24.634564300" Jul 16 04:50:35.094917 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:35.094883 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:35.094917 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:35.094908 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:35.095159 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:35.095018 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2srzm" podUID="8b4ec3ce-70fc-4716-a3c6-5ab123a8be45" Jul 16 04:50:35.095159 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:35.095092 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p97p6" podUID="67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c" Jul 16 04:50:35.095159 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:35.095138 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:35.095278 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:35.095211 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbmmm" podUID="0f651332-08d0-4eb5-b1e8-5d9437727895" Jul 16 04:50:35.231488 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:35.231456 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9lpb5" Jul 16 04:50:35.232374 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:35.232348 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9lpb5" Jul 16 04:50:35.251059 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:35.251030 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" event={"ID":"176b1c0f-1174-47ef-aa5c-25e699c63e02","Type":"ContainerStarted","Data":"92698cb4ae592a4b067473dde10dc5f63c5bcdc6881ea7c5d8d3ab96c1e88b3e"} Jul 16 04:50:36.953087 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:36.952830 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9lpb5" Jul 16 04:50:36.953087 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:36.953084 2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 16 04:50:36.953764 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:36.953354 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9lpb5" Jul 16 04:50:37.095345 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:37.095314 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:37.095485 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:37.095314 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:37.095485 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:37.095423 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbmmm" podUID="0f651332-08d0-4eb5-b1e8-5d9437727895" Jul 16 04:50:37.095611 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:37.095314 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:37.095611 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:37.095491 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p97p6" podUID="67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c" Jul 16 04:50:37.095611 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:37.095553 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2srzm" podUID="8b4ec3ce-70fc-4716-a3c6-5ab123a8be45" Jul 16 04:50:37.258257 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:37.258190 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" event={"ID":"176b1c0f-1174-47ef-aa5c-25e699c63e02","Type":"ContainerStarted","Data":"d07421aa1f438363f5b0eafa6ffe90d739961c43dde38ceadb39ef507e7c59e9"} Jul 16 04:50:37.258423 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:37.258407 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:37.260094 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:37.260070 2561 generic.go:356] "Generic (PLEG): container finished" podID="e18af1c7-b01c-40d1-825b-65cb623e4c5d" containerID="1f1724fa028fd23f0d8ad640c550297a85d66f1ccd2296ef327b3ae9210a1173" exitCode=0 Jul 16 04:50:37.260203 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:37.260153 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hkq2b" event={"ID":"e18af1c7-b01c-40d1-825b-65cb623e4c5d","Type":"ContainerDied","Data":"1f1724fa028fd23f0d8ad640c550297a85d66f1ccd2296ef327b3ae9210a1173"} Jul 16 04:50:37.274397 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:37.274377 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:37.282380 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:37.282345 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" podStartSLOduration=8.614070903 podStartE2EDuration="27.282333357s" totalImagesPullingTime="18.668262454s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:50:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:50:37.280633815 +0000 UTC m=+27.652757021" watchObservedRunningTime="2026-07-16 04:50:37.282333357 +0000 UTC m=+27.654456581" Jul 16 04:50:38.262794 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:38.262752 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:38.263161 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:38.262825 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:38.278133 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:38.278110 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:50:38.611237 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:38.611026 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2srzm"] Jul 16 04:50:38.611417 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:38.611403 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:38.611525 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:38.611504 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2srzm" podUID="8b4ec3ce-70fc-4716-a3c6-5ab123a8be45" Jul 16 04:50:38.611619 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:38.611602 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p97p6"] Jul 16 04:50:38.611718 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:38.611707 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:38.611801 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:38.611783 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p97p6" podUID="67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c" Jul 16 04:50:38.622143 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:38.622119 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jbmmm"] Jul 16 04:50:38.622228 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:38.622216 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:38.622324 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:38.622306 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbmmm" podUID="0f651332-08d0-4eb5-b1e8-5d9437727895" Jul 16 04:50:39.266404 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:39.266373 2561 generic.go:356] "Generic (PLEG): container finished" podID="e18af1c7-b01c-40d1-825b-65cb623e4c5d" containerID="9954834c86435c17a86edbf895a6607f432f5219c9dbbb3610aa6ae2445da2dd" exitCode=0 Jul 16 04:50:39.266834 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:39.266459 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hkq2b" event={"ID":"e18af1c7-b01c-40d1-825b-65cb623e4c5d","Type":"ContainerDied","Data":"9954834c86435c17a86edbf895a6607f432f5219c9dbbb3610aa6ae2445da2dd"} Jul 16 04:50:40.096023 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:40.095997 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:40.096176 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:40.096078 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p97p6" podUID="67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c" Jul 16 04:50:40.096176 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:40.096160 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:40.096312 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:40.096289 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2srzm" podUID="8b4ec3ce-70fc-4716-a3c6-5ab123a8be45" Jul 16 04:50:41.095313 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:41.095280 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:41.095842 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:41.095413 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbmmm" podUID="0f651332-08d0-4eb5-b1e8-5d9437727895" Jul 16 04:50:41.274185 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:41.274152 2561 generic.go:356] "Generic (PLEG): container finished" podID="e18af1c7-b01c-40d1-825b-65cb623e4c5d" containerID="08567cbef028087e3168395efec7713359b745b68c4aedd15eeabcfd7fa5c1c4" exitCode=0 Jul 16 04:50:41.274328 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:41.274202 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hkq2b" event={"ID":"e18af1c7-b01c-40d1-825b-65cb623e4c5d","Type":"ContainerDied","Data":"08567cbef028087e3168395efec7713359b745b68c4aedd15eeabcfd7fa5c1c4"} Jul 16 04:50:42.094897 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:42.094856 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:42.094897 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:42.094894 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:42.095109 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:42.094989 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p97p6" podUID="67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c" Jul 16 04:50:42.095109 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:42.095054 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2srzm" podUID="8b4ec3ce-70fc-4716-a3c6-5ab123a8be45" Jul 16 04:50:43.094928 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:43.094897 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:43.095522 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:43.095025 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbmmm" podUID="0f651332-08d0-4eb5-b1e8-5d9437727895" Jul 16 04:50:43.716531 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:43.716498 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-original-pull-secret\") pod \"global-pull-secret-syncer-p97p6\" (UID: \"67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c\") " pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:43.716771 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:43.716609 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 04:50:43.716771 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:43.716659 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-original-pull-secret podName:67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c nodeName:}" failed. No retries permitted until 2026-07-16 04:51:15.716645609 +0000 UTC m=+66.088768792 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-original-pull-secret") pod "global-pull-secret-syncer-p97p6" (UID: "67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c") : object "kube-system"/"original-pull-secret" not registered Jul 16 04:50:43.816784 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:43.816754 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs\") pod \"network-metrics-daemon-2srzm\" (UID: \"8b4ec3ce-70fc-4716-a3c6-5ab123a8be45\") " pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:43.816926 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:43.816895 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:50:43.816969 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:43.816942 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs podName:8b4ec3ce-70fc-4716-a3c6-5ab123a8be45 nodeName:}" failed. No retries permitted until 2026-07-16 04:51:15.816929695 +0000 UTC m=+66.189052877 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs") pod "network-metrics-daemon-2srzm" (UID: "8b4ec3ce-70fc-4716-a3c6-5ab123a8be45") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:50:43.917203 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:43.917161 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs4tn\" (UniqueName: \"kubernetes.io/projected/0f651332-08d0-4eb5-b1e8-5d9437727895-kube-api-access-gs4tn\") pod \"network-check-target-jbmmm\" (UID: \"0f651332-08d0-4eb5-b1e8-5d9437727895\") " pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:43.917383 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:43.917284 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 04:50:43.917383 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:43.917305 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 04:50:43.917383 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:43.917318 2561 projected.go:196] Error preparing data for projected volume kube-api-access-gs4tn for pod openshift-network-diagnostics/network-check-target-jbmmm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:50:43.917383 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:43.917373 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f651332-08d0-4eb5-b1e8-5d9437727895-kube-api-access-gs4tn podName:0f651332-08d0-4eb5-b1e8-5d9437727895 nodeName:}" failed. No retries permitted until 2026-07-16 04:51:15.917358025 +0000 UTC m=+66.289481224 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-gs4tn" (UniqueName: "kubernetes.io/projected/0f651332-08d0-4eb5-b1e8-5d9437727895-kube-api-access-gs4tn") pod "network-check-target-jbmmm" (UID: "0f651332-08d0-4eb5-b1e8-5d9437727895") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:50:44.095599 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:44.095566 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:44.096050 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:44.095611 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:44.096050 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:44.095738 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p97p6" podUID="67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c" Jul 16 04:50:44.096050 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:44.095879 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2srzm" podUID="8b4ec3ce-70fc-4716-a3c6-5ab123a8be45" Jul 16 04:50:45.095102 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.095066 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:45.095275 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:45.095201 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jbmmm" podUID="0f651332-08d0-4eb5-b1e8-5d9437727895" Jul 16 04:50:45.466140 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.465905 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-187.ec2.internal" event="NodeReady" Jul 16 04:50:45.466473 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.466164 2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 16 04:50:45.506723 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.506678 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-q7ghj"] Jul 16 04:50:45.522544 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.522522 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-n86pm"] Jul 16 04:50:45.522697 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.522659 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q7ghj" Jul 16 04:50:45.525144 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.525124 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dcm9d\"" Jul 16 04:50:45.525276 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.525191 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 16 04:50:45.525343 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.525334 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 16 04:50:45.544917 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.544899 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-q7ghj"] Jul 16 04:50:45.545020 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.544932 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n86pm"] Jul 16 04:50:45.545020 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.545003 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n86pm" Jul 16 04:50:45.547255 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.547226 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 16 04:50:45.547355 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.547324 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2p66f\"" Jul 16 04:50:45.547355 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.547328 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 16 04:50:45.547463 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.547329 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 16 04:50:45.628302 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.628282 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/abe48c82-b51a-47de-9586-6fb499424b6a-config-volume\") pod \"dns-default-q7ghj\" (UID: \"abe48c82-b51a-47de-9586-6fb499424b6a\") " pod="openshift-dns/dns-default-q7ghj" Jul 16 04:50:45.628414 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.628310 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abe48c82-b51a-47de-9586-6fb499424b6a-tmp-dir\") pod \"dns-default-q7ghj\" (UID: \"abe48c82-b51a-47de-9586-6fb499424b6a\") " pod="openshift-dns/dns-default-q7ghj" Jul 16 04:50:45.628414 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.628331 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnsl6\" (UniqueName: \"kubernetes.io/projected/0e460403-6e6e-40d8-9a53-446614aba668-kube-api-access-mnsl6\") pod \"ingress-canary-n86pm\" (UID: \"0e460403-6e6e-40d8-9a53-446614aba668\") " pod="openshift-ingress-canary/ingress-canary-n86pm" Jul 16 04:50:45.628414 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.628347 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6dkp\" (UniqueName: \"kubernetes.io/projected/abe48c82-b51a-47de-9586-6fb499424b6a-kube-api-access-s6dkp\") pod \"dns-default-q7ghj\" (UID: \"abe48c82-b51a-47de-9586-6fb499424b6a\") " pod="openshift-dns/dns-default-q7ghj" Jul 16 04:50:45.628567 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.628419 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e460403-6e6e-40d8-9a53-446614aba668-cert\") pod \"ingress-canary-n86pm\" (UID: \"0e460403-6e6e-40d8-9a53-446614aba668\") " pod="openshift-ingress-canary/ingress-canary-n86pm" Jul 16 04:50:45.628567 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.628446 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls\") pod \"dns-default-q7ghj\" (UID: \"abe48c82-b51a-47de-9586-6fb499424b6a\") " pod="openshift-dns/dns-default-q7ghj" Jul 16 04:50:45.729547 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.729491 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/abe48c82-b51a-47de-9586-6fb499424b6a-config-volume\") pod \"dns-default-q7ghj\" (UID: \"abe48c82-b51a-47de-9586-6fb499424b6a\") " pod="openshift-dns/dns-default-q7ghj" Jul 16 04:50:45.729547 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.729525 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abe48c82-b51a-47de-9586-6fb499424b6a-tmp-dir\") pod \"dns-default-q7ghj\" (UID: \"abe48c82-b51a-47de-9586-6fb499424b6a\") " pod="openshift-dns/dns-default-q7ghj" Jul 16 04:50:45.729547 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.729547 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnsl6\" (UniqueName: \"kubernetes.io/projected/0e460403-6e6e-40d8-9a53-446614aba668-kube-api-access-mnsl6\") pod \"ingress-canary-n86pm\" (UID: \"0e460403-6e6e-40d8-9a53-446614aba668\") " pod="openshift-ingress-canary/ingress-canary-n86pm" Jul 16 04:50:45.729718 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.729669 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6dkp\" (UniqueName: \"kubernetes.io/projected/abe48c82-b51a-47de-9586-6fb499424b6a-kube-api-access-s6dkp\") pod \"dns-default-q7ghj\" (UID: \"abe48c82-b51a-47de-9586-6fb499424b6a\") " pod="openshift-dns/dns-default-q7ghj" Jul 16 04:50:45.729765 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.729738 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e460403-6e6e-40d8-9a53-446614aba668-cert\") pod \"ingress-canary-n86pm\" (UID: \"0e460403-6e6e-40d8-9a53-446614aba668\") " pod="openshift-ingress-canary/ingress-canary-n86pm" Jul 16 04:50:45.729802 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.729771 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls\") pod \"dns-default-q7ghj\" (UID: \"abe48c82-b51a-47de-9586-6fb499424b6a\") " pod="openshift-dns/dns-default-q7ghj" Jul 16 04:50:45.729858 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.729837 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abe48c82-b51a-47de-9586-6fb499424b6a-tmp-dir\") pod \"dns-default-q7ghj\" (UID: \"abe48c82-b51a-47de-9586-6fb499424b6a\") " pod="openshift-dns/dns-default-q7ghj" Jul 16 04:50:45.729909 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:45.729891 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 04:50:45.729955 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:45.729943 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls podName:abe48c82-b51a-47de-9586-6fb499424b6a nodeName:}" failed. No retries permitted until 2026-07-16 04:50:46.229927376 +0000 UTC m=+36.602050560 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls") pod "dns-default-q7ghj" (UID: "abe48c82-b51a-47de-9586-6fb499424b6a") : secret "dns-default-metrics-tls" not found Jul 16 04:50:45.730120 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.730102 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/abe48c82-b51a-47de-9586-6fb499424b6a-config-volume\") pod \"dns-default-q7ghj\" (UID: \"abe48c82-b51a-47de-9586-6fb499424b6a\") " pod="openshift-dns/dns-default-q7ghj" Jul 16 04:50:45.734321 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.734300 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e460403-6e6e-40d8-9a53-446614aba668-cert\") pod \"ingress-canary-n86pm\" (UID: \"0e460403-6e6e-40d8-9a53-446614aba668\") " pod="openshift-ingress-canary/ingress-canary-n86pm" Jul 16 04:50:45.740116 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.740096 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnsl6\" (UniqueName: \"kubernetes.io/projected/0e460403-6e6e-40d8-9a53-446614aba668-kube-api-access-mnsl6\") pod \"ingress-canary-n86pm\" (UID: \"0e460403-6e6e-40d8-9a53-446614aba668\") " pod="openshift-ingress-canary/ingress-canary-n86pm" Jul 16 04:50:45.740190 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.740126 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6dkp\" (UniqueName: \"kubernetes.io/projected/abe48c82-b51a-47de-9586-6fb499424b6a-kube-api-access-s6dkp\") pod \"dns-default-q7ghj\" (UID: \"abe48c82-b51a-47de-9586-6fb499424b6a\") " pod="openshift-dns/dns-default-q7ghj" Jul 16 04:50:45.853053 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:45.853031 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n86pm" Jul 16 04:50:46.041129 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:46.041098 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n86pm"] Jul 16 04:50:46.044792 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:50:46.044767 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e460403_6e6e_40d8_9a53_446614aba668.slice/crio-187b9d75304170c15812b0d7c8e28fce420fbf275c6c428ec0eabb4cd67fc193 WatchSource:0}: Error finding container 187b9d75304170c15812b0d7c8e28fce420fbf275c6c428ec0eabb4cd67fc193: Status 404 returned error can't find the container with id 187b9d75304170c15812b0d7c8e28fce420fbf275c6c428ec0eabb4cd67fc193 Jul 16 04:50:46.094778 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:46.094743 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:50:46.094876 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:46.094741 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:50:46.097270 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:46.097251 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 04:50:46.097380 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:46.097303 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vpdjw\"" Jul 16 04:50:46.097380 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:46.097327 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 04:50:46.232833 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:46.232798 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls\") pod \"dns-default-q7ghj\" (UID: \"abe48c82-b51a-47de-9586-6fb499424b6a\") " pod="openshift-dns/dns-default-q7ghj" Jul 16 04:50:46.232996 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:46.232931 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 04:50:46.233037 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:46.233012 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls podName:abe48c82-b51a-47de-9586-6fb499424b6a nodeName:}" failed. No retries permitted until 2026-07-16 04:50:47.232994311 +0000 UTC m=+37.605117496 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls") pod "dns-default-q7ghj" (UID: "abe48c82-b51a-47de-9586-6fb499424b6a") : secret "dns-default-metrics-tls" not found Jul 16 04:50:46.284978 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:46.284914 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n86pm" event={"ID":"0e460403-6e6e-40d8-9a53-446614aba668","Type":"ContainerStarted","Data":"187b9d75304170c15812b0d7c8e28fce420fbf275c6c428ec0eabb4cd67fc193"} Jul 16 04:50:46.287638 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:46.287612 2561 generic.go:356] "Generic (PLEG): container finished" podID="e18af1c7-b01c-40d1-825b-65cb623e4c5d" containerID="b51dec1efa63a5809b9573525a567389ca38156f2e9fe214d3ace3f1587c8e82" exitCode=0 Jul 16 04:50:46.287751 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:46.287650 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hkq2b" event={"ID":"e18af1c7-b01c-40d1-825b-65cb623e4c5d","Type":"ContainerDied","Data":"b51dec1efa63a5809b9573525a567389ca38156f2e9fe214d3ace3f1587c8e82"} Jul 16 04:50:47.094982 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:47.094796 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:50:47.097825 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:47.097799 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 04:50:47.097957 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:47.097871 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 04:50:47.098867 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:47.098848 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-njf2v\"" Jul 16 04:50:47.241583 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:47.241543 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls\") pod \"dns-default-q7ghj\" (UID: \"abe48c82-b51a-47de-9586-6fb499424b6a\") " pod="openshift-dns/dns-default-q7ghj" Jul 16 04:50:47.241760 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:47.241735 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 04:50:47.241827 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:47.241802 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls podName:abe48c82-b51a-47de-9586-6fb499424b6a nodeName:}" failed. No retries permitted until 2026-07-16 04:50:49.241781247 +0000 UTC m=+39.613904451 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls") pod "dns-default-q7ghj" (UID: "abe48c82-b51a-47de-9586-6fb499424b6a") : secret "dns-default-metrics-tls" not found Jul 16 04:50:47.295473 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:47.295441 2561 generic.go:356] "Generic (PLEG): container finished" podID="e18af1c7-b01c-40d1-825b-65cb623e4c5d" containerID="95e9d1daad4099b9dc1a06958e58464f1b81fe01a2b4a65d24937447301e4f7e" exitCode=0 Jul 16 04:50:47.295662 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:47.295513 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hkq2b" event={"ID":"e18af1c7-b01c-40d1-825b-65cb623e4c5d","Type":"ContainerDied","Data":"95e9d1daad4099b9dc1a06958e58464f1b81fe01a2b4a65d24937447301e4f7e"} Jul 16 04:50:48.302493 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:48.302439 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hkq2b" event={"ID":"e18af1c7-b01c-40d1-825b-65cb623e4c5d","Type":"ContainerStarted","Data":"d945874e2299b74f3307709f66d535637ace9e1467c0acc8b576a6338c1e094c"} Jul 16 04:50:49.258669 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:49.258632 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls\") pod \"dns-default-q7ghj\" (UID: \"abe48c82-b51a-47de-9586-6fb499424b6a\") " pod="openshift-dns/dns-default-q7ghj" Jul 16 04:50:49.258861 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:49.258806 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 04:50:49.258904 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:49.258869 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls podName:abe48c82-b51a-47de-9586-6fb499424b6a nodeName:}" failed. No retries permitted until 2026-07-16 04:50:53.258854 +0000 UTC m=+43.630977183 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls") pod "dns-default-q7ghj" (UID: "abe48c82-b51a-47de-9586-6fb499424b6a") : secret "dns-default-metrics-tls" not found Jul 16 04:50:49.305806 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:49.305779 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n86pm" event={"ID":"0e460403-6e6e-40d8-9a53-446614aba668","Type":"ContainerStarted","Data":"79411ec89adc3c3ee7974f7b2a4f850e2ad7d82ffe29e14e5a0c219c54161138"} Jul 16 04:50:49.320002 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:49.319952 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hkq2b" podStartSLOduration=9.423251424 podStartE2EDuration="39.319932962s" totalImagesPullingTime="29.896681538s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:50:10 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:50:48.325575317 +0000 UTC m=+38.697698541" watchObservedRunningTime="2026-07-16 04:50:49.319932962 +0000 UTC m=+39.692056168" Jul 16 04:50:49.320336 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:49.320315 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-n86pm" podStartSLOduration=1.6154362230000001 podStartE2EDuration="4.320306908s" totalImagesPullingTime="2.704870685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:50:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:50:49.319794378 +0000 UTC m=+39.691917584" watchObservedRunningTime="2026-07-16 04:50:49.320306908 +0000 UTC m=+39.692430113" Jul 16 04:50:53.285868 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:50:53.285824 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls\") pod \"dns-default-q7ghj\" (UID: \"abe48c82-b51a-47de-9586-6fb499424b6a\") " pod="openshift-dns/dns-default-q7ghj" Jul 16 04:50:53.286281 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:53.285980 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 04:50:53.286281 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:50:53.286045 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls podName:abe48c82-b51a-47de-9586-6fb499424b6a nodeName:}" failed. No retries permitted until 2026-07-16 04:51:01.286029607 +0000 UTC m=+51.658152789 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls") pod "dns-default-q7ghj" (UID: "abe48c82-b51a-47de-9586-6fb499424b6a") : secret "dns-default-metrics-tls" not found Jul 16 04:51:01.340659 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:01.340621 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls\") pod \"dns-default-q7ghj\" (UID: \"abe48c82-b51a-47de-9586-6fb499424b6a\") " pod="openshift-dns/dns-default-q7ghj" Jul 16 04:51:01.341144 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:51:01.340743 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 04:51:01.341144 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:51:01.340790 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls podName:abe48c82-b51a-47de-9586-6fb499424b6a nodeName:}" failed. No retries permitted until 2026-07-16 04:51:17.340777901 +0000 UTC m=+67.712901085 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls") pod "dns-default-q7ghj" (UID: "abe48c82-b51a-47de-9586-6fb499424b6a") : secret "dns-default-metrics-tls" not found Jul 16 04:51:10.278838 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:10.278810 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bsqdp" Jul 16 04:51:15.734070 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:15.734033 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-original-pull-secret\") pod \"global-pull-secret-syncer-p97p6\" (UID: \"67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c\") " pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:51:15.736660 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:15.736643 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 04:51:15.747925 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:15.747894 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c-original-pull-secret\") pod \"global-pull-secret-syncer-p97p6\" (UID: \"67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c\") " pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:51:15.805036 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:15.805008 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p97p6" Jul 16 04:51:15.834475 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:15.834457 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs\") pod \"network-metrics-daemon-2srzm\" (UID: \"8b4ec3ce-70fc-4716-a3c6-5ab123a8be45\") " pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:51:15.837150 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:15.837135 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 04:51:15.845079 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:51:15.845064 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 16 04:51:15.845148 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:51:15.845139 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs podName:8b4ec3ce-70fc-4716-a3c6-5ab123a8be45 nodeName:}" failed. No retries permitted until 2026-07-16 04:52:19.845100321 +0000 UTC m=+130.217223504 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs") pod "network-metrics-daemon-2srzm" (UID: "8b4ec3ce-70fc-4716-a3c6-5ab123a8be45") : secret "metrics-daemon-secret" not found Jul 16 04:51:15.935177 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:15.935144 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs4tn\" (UniqueName: \"kubernetes.io/projected/0f651332-08d0-4eb5-b1e8-5d9437727895-kube-api-access-gs4tn\") pod \"network-check-target-jbmmm\" (UID: \"0f651332-08d0-4eb5-b1e8-5d9437727895\") " pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:51:15.937831 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:15.937807 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 04:51:15.947890 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:15.947869 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 04:51:15.959026 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:15.959007 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs4tn\" (UniqueName: \"kubernetes.io/projected/0f651332-08d0-4eb5-b1e8-5d9437727895-kube-api-access-gs4tn\") pod \"network-check-target-jbmmm\" (UID: \"0f651332-08d0-4eb5-b1e8-5d9437727895\") " pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:51:15.963042 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:15.963010 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p97p6"] Jul 16 04:51:15.965939 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:51:15.965920 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67ee4d7c_8ae3_4f53_9a97_84a80cf5d67c.slice/crio-89b3240514dc3aeac3f91dbc4c2e72109ffd960f624f3a34b11110946f081228 WatchSource:0}: Error finding container 89b3240514dc3aeac3f91dbc4c2e72109ffd960f624f3a34b11110946f081228: Status 404 returned error can't find the container with id 89b3240514dc3aeac3f91dbc4c2e72109ffd960f624f3a34b11110946f081228 Jul 16 04:51:16.208853 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:16.208827 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-njf2v\"" Jul 16 04:51:16.216895 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:16.216872 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:51:16.363366 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:16.363338 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jbmmm"] Jul 16 04:51:16.364553 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:16.364528 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p97p6" event={"ID":"67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c","Type":"ContainerStarted","Data":"89b3240514dc3aeac3f91dbc4c2e72109ffd960f624f3a34b11110946f081228"} Jul 16 04:51:16.368046 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:51:16.368025 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f651332_08d0_4eb5_b1e8_5d9437727895.slice/crio-a8c32f15b3e739a78ca5d28105866a9188ffad75232267558661d89864f14519 WatchSource:0}: Error finding container a8c32f15b3e739a78ca5d28105866a9188ffad75232267558661d89864f14519: Status 404 returned error can't find the container with id a8c32f15b3e739a78ca5d28105866a9188ffad75232267558661d89864f14519 Jul 16 04:51:17.345864 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:17.345759 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls\") pod \"dns-default-q7ghj\" (UID: \"abe48c82-b51a-47de-9586-6fb499424b6a\") " pod="openshift-dns/dns-default-q7ghj" Jul 16 04:51:17.346282 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:51:17.345892 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 04:51:17.346282 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:51:17.346018 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls podName:abe48c82-b51a-47de-9586-6fb499424b6a nodeName:}" failed. No retries permitted until 2026-07-16 04:51:49.345995538 +0000 UTC m=+99.718118735 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls") pod "dns-default-q7ghj" (UID: "abe48c82-b51a-47de-9586-6fb499424b6a") : secret "dns-default-metrics-tls" not found Jul 16 04:51:17.369633 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:17.369597 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jbmmm" event={"ID":"0f651332-08d0-4eb5-b1e8-5d9437727895","Type":"ContainerStarted","Data":"a8c32f15b3e739a78ca5d28105866a9188ffad75232267558661d89864f14519"} Jul 16 04:51:21.382344 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:21.382311 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jbmmm" event={"ID":"0f651332-08d0-4eb5-b1e8-5d9437727895","Type":"ContainerStarted","Data":"ad591416853fb5f5f33dc883415b498a5f4fc40abde5754fb503a2703bf27e51"} Jul 16 04:51:21.382783 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:21.382373 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:51:21.383785 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:21.383762 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p97p6" event={"ID":"67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c","Type":"ContainerStarted","Data":"d36815508296b4c2e97d1ae727f0ce14a511573264eb5249f11748542520397c"} Jul 16 04:51:21.396343 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:21.396298 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jbmmm" podStartSLOduration=67.168067863 podStartE2EDuration="1m11.396286867s" totalImagesPullingTime="4.228219004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:50:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:51:21.396186823 +0000 UTC m=+71.768310042" watchObservedRunningTime="2026-07-16 04:51:21.396286867 +0000 UTC m=+71.768410050" Jul 16 04:51:29.916784 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:29.916731 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-p97p6" podStartSLOduration=75.283290708 podStartE2EDuration="1m19.916716437s" totalImagesPullingTime="4.633425729s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:50:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:51:21.408835709 +0000 UTC m=+71.780958935" watchObservedRunningTime="2026-07-16 04:51:29.916716437 +0000 UTC m=+80.288839641" Jul 16 04:51:29.917376 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:29.917358 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b4d44bc56-v9b4t"] Jul 16 04:51:29.920063 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:29.920049 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b4d44bc56-v9b4t" Jul 16 04:51:29.922712 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:29.922672 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 16 04:51:29.923798 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:29.923777 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 16 04:51:29.923895 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:29.923816 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-pqvx7\"" Jul 16 04:51:29.923895 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:29.923816 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 16 04:51:29.923895 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:29.923875 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 16 04:51:29.929635 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:29.929613 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b4d44bc56-v9b4t"] Jul 16 04:51:29.954359 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:29.954339 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm"] Jul 16 04:51:29.957144 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:29.957130 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" Jul 16 04:51:29.959498 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:29.959482 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 16 04:51:29.959599 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:29.959507 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 16 04:51:29.959599 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:29.959574 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 16 04:51:29.959712 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:29.959576 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 16 04:51:29.967664 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:29.967637 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm"] Jul 16 04:51:30.027745 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.027718 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj2zl\" (UniqueName: \"kubernetes.io/projected/655b05a6-782d-4101-b5da-f11d342513ba-kube-api-access-qj2zl\") pod \"managed-serviceaccount-addon-agent-5b4d44bc56-v9b4t\" (UID: \"655b05a6-782d-4101-b5da-f11d342513ba\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b4d44bc56-v9b4t" Jul 16 04:51:30.027874 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.027755 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/655b05a6-782d-4101-b5da-f11d342513ba-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b4d44bc56-v9b4t\" (UID: \"655b05a6-782d-4101-b5da-f11d342513ba\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b4d44bc56-v9b4t" Jul 16 04:51:30.128384 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.128358 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj2zl\" (UniqueName: \"kubernetes.io/projected/655b05a6-782d-4101-b5da-f11d342513ba-kube-api-access-qj2zl\") pod \"managed-serviceaccount-addon-agent-5b4d44bc56-v9b4t\" (UID: \"655b05a6-782d-4101-b5da-f11d342513ba\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b4d44bc56-v9b4t" Jul 16 04:51:30.128494 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.128388 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/655b05a6-782d-4101-b5da-f11d342513ba-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b4d44bc56-v9b4t\" (UID: \"655b05a6-782d-4101-b5da-f11d342513ba\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b4d44bc56-v9b4t" Jul 16 04:51:30.128494 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.128411 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d4b8d332-7841-435c-b0fb-8a6ddf71aec1-ca\") pod \"cluster-proxy-proxy-agent-6f577b6848-wxngm\" (UID: \"d4b8d332-7841-435c-b0fb-8a6ddf71aec1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" Jul 16 04:51:30.128494 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.128426 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d4b8d332-7841-435c-b0fb-8a6ddf71aec1-hub\") pod \"cluster-proxy-proxy-agent-6f577b6848-wxngm\" (UID: \"d4b8d332-7841-435c-b0fb-8a6ddf71aec1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" Jul 16 04:51:30.128494 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.128480 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b8d332-7841-435c-b0fb-8a6ddf71aec1-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6f577b6848-wxngm\" (UID: \"d4b8d332-7841-435c-b0fb-8a6ddf71aec1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" Jul 16 04:51:30.128660 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.128557 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm7fd\" (UniqueName: \"kubernetes.io/projected/d4b8d332-7841-435c-b0fb-8a6ddf71aec1-kube-api-access-jm7fd\") pod \"cluster-proxy-proxy-agent-6f577b6848-wxngm\" (UID: \"d4b8d332-7841-435c-b0fb-8a6ddf71aec1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" Jul 16 04:51:30.128660 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.128589 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d4b8d332-7841-435c-b0fb-8a6ddf71aec1-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6f577b6848-wxngm\" (UID: \"d4b8d332-7841-435c-b0fb-8a6ddf71aec1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" Jul 16 04:51:30.128660 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.128630 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d4b8d332-7841-435c-b0fb-8a6ddf71aec1-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6f577b6848-wxngm\" (UID: \"d4b8d332-7841-435c-b0fb-8a6ddf71aec1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" Jul 16 04:51:30.131020 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.130995 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/655b05a6-782d-4101-b5da-f11d342513ba-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5b4d44bc56-v9b4t\" (UID: \"655b05a6-782d-4101-b5da-f11d342513ba\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b4d44bc56-v9b4t" Jul 16 04:51:30.135571 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.135550 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj2zl\" (UniqueName: \"kubernetes.io/projected/655b05a6-782d-4101-b5da-f11d342513ba-kube-api-access-qj2zl\") pod \"managed-serviceaccount-addon-agent-5b4d44bc56-v9b4t\" (UID: \"655b05a6-782d-4101-b5da-f11d342513ba\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b4d44bc56-v9b4t" Jul 16 04:51:30.229419 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.229342 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d4b8d332-7841-435c-b0fb-8a6ddf71aec1-ca\") pod \"cluster-proxy-proxy-agent-6f577b6848-wxngm\" (UID: \"d4b8d332-7841-435c-b0fb-8a6ddf71aec1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" Jul 16 04:51:30.229419 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.229374 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d4b8d332-7841-435c-b0fb-8a6ddf71aec1-hub\") pod \"cluster-proxy-proxy-agent-6f577b6848-wxngm\" (UID: \"d4b8d332-7841-435c-b0fb-8a6ddf71aec1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" Jul 16 04:51:30.229419 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.229391 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b8d332-7841-435c-b0fb-8a6ddf71aec1-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6f577b6848-wxngm\" (UID: \"d4b8d332-7841-435c-b0fb-8a6ddf71aec1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" Jul 16 04:51:30.229419 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.229410 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jm7fd\" (UniqueName: \"kubernetes.io/projected/d4b8d332-7841-435c-b0fb-8a6ddf71aec1-kube-api-access-jm7fd\") pod \"cluster-proxy-proxy-agent-6f577b6848-wxngm\" (UID: \"d4b8d332-7841-435c-b0fb-8a6ddf71aec1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" Jul 16 04:51:30.229707 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.229461 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d4b8d332-7841-435c-b0fb-8a6ddf71aec1-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6f577b6848-wxngm\" (UID: \"d4b8d332-7841-435c-b0fb-8a6ddf71aec1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" Jul 16 04:51:30.229707 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.229535 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d4b8d332-7841-435c-b0fb-8a6ddf71aec1-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6f577b6848-wxngm\" (UID: \"d4b8d332-7841-435c-b0fb-8a6ddf71aec1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" Jul 16 04:51:30.230174 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.230082 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b8d332-7841-435c-b0fb-8a6ddf71aec1-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6f577b6848-wxngm\" (UID: \"d4b8d332-7841-435c-b0fb-8a6ddf71aec1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" Jul 16 04:51:30.231901 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.231880 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d4b8d332-7841-435c-b0fb-8a6ddf71aec1-ca\") pod \"cluster-proxy-proxy-agent-6f577b6848-wxngm\" (UID: \"d4b8d332-7841-435c-b0fb-8a6ddf71aec1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" Jul 16 04:51:30.232076 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.232057 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d4b8d332-7841-435c-b0fb-8a6ddf71aec1-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6f577b6848-wxngm\" (UID: \"d4b8d332-7841-435c-b0fb-8a6ddf71aec1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" Jul 16 04:51:30.232450 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.232435 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d4b8d332-7841-435c-b0fb-8a6ddf71aec1-hub\") pod \"cluster-proxy-proxy-agent-6f577b6848-wxngm\" (UID: \"d4b8d332-7841-435c-b0fb-8a6ddf71aec1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" Jul 16 04:51:30.232656 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.232636 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d4b8d332-7841-435c-b0fb-8a6ddf71aec1-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6f577b6848-wxngm\" (UID: \"d4b8d332-7841-435c-b0fb-8a6ddf71aec1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" Jul 16 04:51:30.236772 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.236751 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm7fd\" (UniqueName: \"kubernetes.io/projected/d4b8d332-7841-435c-b0fb-8a6ddf71aec1-kube-api-access-jm7fd\") pod \"cluster-proxy-proxy-agent-6f577b6848-wxngm\" (UID: \"d4b8d332-7841-435c-b0fb-8a6ddf71aec1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" Jul 16 04:51:30.243651 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.243571 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b4d44bc56-v9b4t" Jul 16 04:51:30.267472 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.267448 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" Jul 16 04:51:30.370853 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.370826 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b4d44bc56-v9b4t"] Jul 16 04:51:30.373542 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:51:30.373513 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod655b05a6_782d_4101_b5da_f11d342513ba.slice/crio-4a6766993f055e9c237fab3aeefb1e2b55da6d6cf1a3cf41a6a2d16fbcaa50e1 WatchSource:0}: Error finding container 4a6766993f055e9c237fab3aeefb1e2b55da6d6cf1a3cf41a6a2d16fbcaa50e1: Status 404 returned error can't find the container with id 4a6766993f055e9c237fab3aeefb1e2b55da6d6cf1a3cf41a6a2d16fbcaa50e1 Jul 16 04:51:30.396471 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.396452 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm"] Jul 16 04:51:30.398598 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:51:30.398579 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd4b8d332_7841_435c_b0fb_8a6ddf71aec1.slice/crio-f92bfd226ae3c2d3a48a662ab2dc60e6ed9889371aa1f9178a29b0a0f39822ac WatchSource:0}: Error finding container f92bfd226ae3c2d3a48a662ab2dc60e6ed9889371aa1f9178a29b0a0f39822ac: Status 404 returned error can't find the container with id f92bfd226ae3c2d3a48a662ab2dc60e6ed9889371aa1f9178a29b0a0f39822ac Jul 16 04:51:30.406890 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.406864 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" event={"ID":"d4b8d332-7841-435c-b0fb-8a6ddf71aec1","Type":"ContainerStarted","Data":"f92bfd226ae3c2d3a48a662ab2dc60e6ed9889371aa1f9178a29b0a0f39822ac"} Jul 16 04:51:30.407824 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:30.407800 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b4d44bc56-v9b4t" event={"ID":"655b05a6-782d-4101-b5da-f11d342513ba","Type":"ContainerStarted","Data":"4a6766993f055e9c237fab3aeefb1e2b55da6d6cf1a3cf41a6a2d16fbcaa50e1"} Jul 16 04:51:35.424106 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:35.424066 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b4d44bc56-v9b4t" event={"ID":"655b05a6-782d-4101-b5da-f11d342513ba","Type":"ContainerStarted","Data":"0c15584a7df0e0a24bde90783f4d6e8dfcbe3396fecf6aa5ed02118f71ba61ac"} Jul 16 04:51:35.426088 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:35.426067 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" event={"ID":"d4b8d332-7841-435c-b0fb-8a6ddf71aec1","Type":"ContainerStarted","Data":"b1ea6a20da8d88fb945f05dae6aae7daa4a62fcc82777fc526c2f6f2505d9c3c"} Jul 16 04:51:35.426195 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:35.426098 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" event={"ID":"d4b8d332-7841-435c-b0fb-8a6ddf71aec1","Type":"ContainerStarted","Data":"574807a42dcd99b4d2ab4d23e6e413b3201a18395c32dc64cadb45579dcc163b"} Jul 16 04:51:35.426195 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:35.426111 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" event={"ID":"d4b8d332-7841-435c-b0fb-8a6ddf71aec1","Type":"ContainerStarted","Data":"78023d154e9d14cd334435c5d5777d5a02c33e0a8fc2963b3d47265a7cf09ed9"} Jul 16 04:51:35.438170 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:35.438131 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b4d44bc56-v9b4t" podStartSLOduration=2.368914058 podStartE2EDuration="6.438118895s" totalImagesPullingTime="4.069204837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:51:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:51:35.437486508 +0000 UTC m=+85.809609712" watchObservedRunningTime="2026-07-16 04:51:35.438118895 +0000 UTC m=+85.810242102" Jul 16 04:51:35.453421 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:35.453380 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" podStartSLOduration=2.397020732 podStartE2EDuration="6.453369536s" totalImagesPullingTime="4.056348804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:51:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:51:35.45263464 +0000 UTC m=+85.824757839" watchObservedRunningTime="2026-07-16 04:51:35.453369536 +0000 UTC m=+85.825492740" Jul 16 04:51:49.357812 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:49.357766 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls\") pod \"dns-default-q7ghj\" (UID: \"abe48c82-b51a-47de-9586-6fb499424b6a\") " pod="openshift-dns/dns-default-q7ghj" Jul 16 04:51:49.358190 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:51:49.357875 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 04:51:49.358190 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:51:49.357933 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls podName:abe48c82-b51a-47de-9586-6fb499424b6a nodeName:}" failed. No retries permitted until 2026-07-16 04:52:53.357919291 +0000 UTC m=+163.730042475 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls") pod "dns-default-q7ghj" (UID: "abe48c82-b51a-47de-9586-6fb499424b6a") : secret "dns-default-metrics-tls" not found Jul 16 04:51:52.388438 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:51:52.388408 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jbmmm" Jul 16 04:52:19.861810 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:52:19.861772 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs\") pod \"network-metrics-daemon-2srzm\" (UID: \"8b4ec3ce-70fc-4716-a3c6-5ab123a8be45\") " pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:52:19.862337 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:52:19.861917 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 16 04:52:19.862337 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:52:19.862001 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs podName:8b4ec3ce-70fc-4716-a3c6-5ab123a8be45 nodeName:}" failed. No retries permitted until 2026-07-16 04:54:21.861981624 +0000 UTC m=+252.234104812 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs") pod "network-metrics-daemon-2srzm" (UID: "8b4ec3ce-70fc-4716-a3c6-5ab123a8be45") : secret "metrics-daemon-secret" not found Jul 16 04:52:48.532859 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:52:48.532772 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-q7ghj" podUID="abe48c82-b51a-47de-9586-6fb499424b6a" Jul 16 04:52:48.629696 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:52:48.629657 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q7ghj" Jul 16 04:52:49.110657 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:52:49.110613 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-2srzm" podUID="8b4ec3ce-70fc-4716-a3c6-5ab123a8be45" Jul 16 04:52:53.375055 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:52:53.375021 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls\") pod \"dns-default-q7ghj\" (UID: \"abe48c82-b51a-47de-9586-6fb499424b6a\") " pod="openshift-dns/dns-default-q7ghj" Jul 16 04:52:53.375431 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:52:53.375123 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 04:52:53.375431 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:52:53.375171 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls podName:abe48c82-b51a-47de-9586-6fb499424b6a nodeName:}" failed. No retries permitted until 2026-07-16 04:54:55.375158679 +0000 UTC m=+285.747281862 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls") pod "dns-default-q7ghj" (UID: "abe48c82-b51a-47de-9586-6fb499424b6a") : secret "dns-default-metrics-tls" not found Jul 16 04:53:00.095970 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:53:00.095927 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:53:34.754508 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:53:34.754480 2561 generic.go:356] "Generic (PLEG): container finished" podID="655b05a6-782d-4101-b5da-f11d342513ba" containerID="0c15584a7df0e0a24bde90783f4d6e8dfcbe3396fecf6aa5ed02118f71ba61ac" exitCode=255 Jul 16 04:53:34.754829 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:53:34.754543 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b4d44bc56-v9b4t" event={"ID":"655b05a6-782d-4101-b5da-f11d342513ba","Type":"ContainerDied","Data":"0c15584a7df0e0a24bde90783f4d6e8dfcbe3396fecf6aa5ed02118f71ba61ac"} Jul 16 04:53:34.754829 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:53:34.754799 2561 scope.go:117] "RemoveContainer" containerID="0c15584a7df0e0a24bde90783f4d6e8dfcbe3396fecf6aa5ed02118f71ba61ac" Jul 16 04:53:35.758555 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:53:35.758518 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5b4d44bc56-v9b4t" event={"ID":"655b05a6-782d-4101-b5da-f11d342513ba","Type":"ContainerStarted","Data":"019c4e8217bdc1ac3dc854ea18c4814addfd95375f9b30175ffb1769c65a6aa1"} Jul 16 04:54:21.483476 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:21.483445 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bmns9"] Jul 16 04:54:21.486459 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:21.486432 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:21.489000 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:21.488977 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 16 04:54:21.489120 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:21.488977 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 16 04:54:21.489120 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:21.489033 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q2w4b\"" Jul 16 04:54:21.489120 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:21.489069 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 16 04:54:21.490215 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:21.490197 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 16 04:54:21.498176 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:21.498154 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bmns9"] Jul 16 04:54:21.648734 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:21.648706 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42adc88e-a3b3-42eb-a476-d796fb65e81e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bmns9\" (UID: \"42adc88e-a3b3-42eb-a476-d796fb65e81e\") " pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:21.648873 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:21.648741 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42adc88e-a3b3-42eb-a476-d796fb65e81e-data-volume\") pod \"insights-runtime-extractor-bmns9\" (UID: \"42adc88e-a3b3-42eb-a476-d796fb65e81e\") " pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:21.648873 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:21.648759 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42adc88e-a3b3-42eb-a476-d796fb65e81e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bmns9\" (UID: \"42adc88e-a3b3-42eb-a476-d796fb65e81e\") " pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:21.648873 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:21.648831 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42adc88e-a3b3-42eb-a476-d796fb65e81e-crio-socket\") pod \"insights-runtime-extractor-bmns9\" (UID: \"42adc88e-a3b3-42eb-a476-d796fb65e81e\") " pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:21.648873 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:21.648866 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdwbk\" (UniqueName: \"kubernetes.io/projected/42adc88e-a3b3-42eb-a476-d796fb65e81e-kube-api-access-kdwbk\") pod \"insights-runtime-extractor-bmns9\" (UID: \"42adc88e-a3b3-42eb-a476-d796fb65e81e\") " pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:21.749576 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:21.749522 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42adc88e-a3b3-42eb-a476-d796fb65e81e-data-volume\") pod \"insights-runtime-extractor-bmns9\" (UID: \"42adc88e-a3b3-42eb-a476-d796fb65e81e\") " pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:21.749576 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:21.749547 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42adc88e-a3b3-42eb-a476-d796fb65e81e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bmns9\" (UID: \"42adc88e-a3b3-42eb-a476-d796fb65e81e\") " pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:21.749576 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:21.749570 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42adc88e-a3b3-42eb-a476-d796fb65e81e-crio-socket\") pod \"insights-runtime-extractor-bmns9\" (UID: \"42adc88e-a3b3-42eb-a476-d796fb65e81e\") " pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:21.749769 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:21.749588 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdwbk\" (UniqueName: \"kubernetes.io/projected/42adc88e-a3b3-42eb-a476-d796fb65e81e-kube-api-access-kdwbk\") pod \"insights-runtime-extractor-bmns9\" (UID: \"42adc88e-a3b3-42eb-a476-d796fb65e81e\") " pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:21.749769 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:21.749654 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42adc88e-a3b3-42eb-a476-d796fb65e81e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bmns9\" (UID: \"42adc88e-a3b3-42eb-a476-d796fb65e81e\") " pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:21.749769 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:21.749662 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42adc88e-a3b3-42eb-a476-d796fb65e81e-crio-socket\") pod \"insights-runtime-extractor-bmns9\" (UID: \"42adc88e-a3b3-42eb-a476-d796fb65e81e\") " pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:21.749769 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:21.749668 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 04:54:21.749769 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:21.749757 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 04:54:21.749944 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:21.749772 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42adc88e-a3b3-42eb-a476-d796fb65e81e-kube-rbac-proxy-cm podName:42adc88e-a3b3-42eb-a476-d796fb65e81e nodeName:}" failed. No retries permitted until 2026-07-16 04:54:22.249753827 +0000 UTC m=+252.621877009 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42adc88e-a3b3-42eb-a476-d796fb65e81e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bmns9" (UID: "42adc88e-a3b3-42eb-a476-d796fb65e81e") : configmap "kube-rbac-proxy" not found Jul 16 04:54:21.749944 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:21.749800 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42adc88e-a3b3-42eb-a476-d796fb65e81e-insights-runtime-extractor-tls podName:42adc88e-a3b3-42eb-a476-d796fb65e81e nodeName:}" failed. No retries permitted until 2026-07-16 04:54:22.249788675 +0000 UTC m=+252.621911857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42adc88e-a3b3-42eb-a476-d796fb65e81e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bmns9" (UID: "42adc88e-a3b3-42eb-a476-d796fb65e81e") : secret "insights-runtime-extractor-tls" not found Jul 16 04:54:21.749944 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:21.749826 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42adc88e-a3b3-42eb-a476-d796fb65e81e-data-volume\") pod \"insights-runtime-extractor-bmns9\" (UID: \"42adc88e-a3b3-42eb-a476-d796fb65e81e\") " pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:21.758705 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:21.758673 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdwbk\" (UniqueName: \"kubernetes.io/projected/42adc88e-a3b3-42eb-a476-d796fb65e81e-kube-api-access-kdwbk\") pod \"insights-runtime-extractor-bmns9\" (UID: \"42adc88e-a3b3-42eb-a476-d796fb65e81e\") " pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:21.951321 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:21.951296 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs\") pod \"network-metrics-daemon-2srzm\" (UID: \"8b4ec3ce-70fc-4716-a3c6-5ab123a8be45\") " pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:54:21.951429 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:21.951390 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 16 04:54:21.951467 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:21.951441 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs podName:8b4ec3ce-70fc-4716-a3c6-5ab123a8be45 nodeName:}" failed. No retries permitted until 2026-07-16 04:56:23.951428758 +0000 UTC m=+374.323551940 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs") pod "network-metrics-daemon-2srzm" (UID: "8b4ec3ce-70fc-4716-a3c6-5ab123a8be45") : secret "metrics-daemon-secret" not found Jul 16 04:54:22.253406 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:22.253378 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42adc88e-a3b3-42eb-a476-d796fb65e81e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bmns9\" (UID: \"42adc88e-a3b3-42eb-a476-d796fb65e81e\") " pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:22.253551 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:22.253413 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42adc88e-a3b3-42eb-a476-d796fb65e81e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bmns9\" (UID: \"42adc88e-a3b3-42eb-a476-d796fb65e81e\") " pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:22.253551 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:22.253474 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 04:54:22.253551 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:22.253519 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42adc88e-a3b3-42eb-a476-d796fb65e81e-kube-rbac-proxy-cm podName:42adc88e-a3b3-42eb-a476-d796fb65e81e nodeName:}" failed. No retries permitted until 2026-07-16 04:54:23.253507067 +0000 UTC m=+253.625630250 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42adc88e-a3b3-42eb-a476-d796fb65e81e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bmns9" (UID: "42adc88e-a3b3-42eb-a476-d796fb65e81e") : configmap "kube-rbac-proxy" not found Jul 16 04:54:22.253551 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:22.253533 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 04:54:22.253743 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:22.253587 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42adc88e-a3b3-42eb-a476-d796fb65e81e-insights-runtime-extractor-tls podName:42adc88e-a3b3-42eb-a476-d796fb65e81e nodeName:}" failed. No retries permitted until 2026-07-16 04:54:23.253571713 +0000 UTC m=+253.625694896 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42adc88e-a3b3-42eb-a476-d796fb65e81e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bmns9" (UID: "42adc88e-a3b3-42eb-a476-d796fb65e81e") : secret "insights-runtime-extractor-tls" not found Jul 16 04:54:23.260163 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:23.260118 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42adc88e-a3b3-42eb-a476-d796fb65e81e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bmns9\" (UID: \"42adc88e-a3b3-42eb-a476-d796fb65e81e\") " pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:23.260658 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:23.260176 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42adc88e-a3b3-42eb-a476-d796fb65e81e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bmns9\" (UID: \"42adc88e-a3b3-42eb-a476-d796fb65e81e\") " pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:23.260658 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:23.260269 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 04:54:23.260658 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:23.260295 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 04:54:23.260658 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:23.260347 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42adc88e-a3b3-42eb-a476-d796fb65e81e-insights-runtime-extractor-tls podName:42adc88e-a3b3-42eb-a476-d796fb65e81e nodeName:}" failed. No retries permitted until 2026-07-16 04:54:25.260330807 +0000 UTC m=+255.632453993 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42adc88e-a3b3-42eb-a476-d796fb65e81e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bmns9" (UID: "42adc88e-a3b3-42eb-a476-d796fb65e81e") : secret "insights-runtime-extractor-tls" not found Jul 16 04:54:23.260658 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:23.260363 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42adc88e-a3b3-42eb-a476-d796fb65e81e-kube-rbac-proxy-cm podName:42adc88e-a3b3-42eb-a476-d796fb65e81e nodeName:}" failed. No retries permitted until 2026-07-16 04:54:25.260354937 +0000 UTC m=+255.632478120 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42adc88e-a3b3-42eb-a476-d796fb65e81e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bmns9" (UID: "42adc88e-a3b3-42eb-a476-d796fb65e81e") : configmap "kube-rbac-proxy" not found Jul 16 04:54:23.661737 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:23.661713 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vkqpj_076bc066-399e-417e-a245-14833a939ca8/dns-node-resolver/0.log" Jul 16 04:54:24.263652 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:24.263627 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2lr5q_ffc2facd-a805-43cc-91d2-e35ffff459fd/node-ca/0.log" Jul 16 04:54:25.272852 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:25.272817 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42adc88e-a3b3-42eb-a476-d796fb65e81e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bmns9\" (UID: \"42adc88e-a3b3-42eb-a476-d796fb65e81e\") " pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:25.273329 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:25.272862 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42adc88e-a3b3-42eb-a476-d796fb65e81e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bmns9\" (UID: \"42adc88e-a3b3-42eb-a476-d796fb65e81e\") " pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:25.273329 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:25.272948 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 04:54:25.273329 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:25.272958 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 04:54:25.273329 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:25.273014 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42adc88e-a3b3-42eb-a476-d796fb65e81e-kube-rbac-proxy-cm podName:42adc88e-a3b3-42eb-a476-d796fb65e81e nodeName:}" failed. No retries permitted until 2026-07-16 04:54:29.272996257 +0000 UTC m=+259.645119441 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42adc88e-a3b3-42eb-a476-d796fb65e81e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bmns9" (UID: "42adc88e-a3b3-42eb-a476-d796fb65e81e") : configmap "kube-rbac-proxy" not found Jul 16 04:54:25.273329 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:25.273034 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42adc88e-a3b3-42eb-a476-d796fb65e81e-insights-runtime-extractor-tls podName:42adc88e-a3b3-42eb-a476-d796fb65e81e nodeName:}" failed. No retries permitted until 2026-07-16 04:54:29.273025233 +0000 UTC m=+259.645148418 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42adc88e-a3b3-42eb-a476-d796fb65e81e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bmns9" (UID: "42adc88e-a3b3-42eb-a476-d796fb65e81e") : secret "insights-runtime-extractor-tls" not found Jul 16 04:54:25.464679 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:25.464649 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n86pm_0e460403-6e6e-40d8-9a53-446614aba668/serve-healthcheck-canary/0.log" Jul 16 04:54:29.296742 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:29.296706 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42adc88e-a3b3-42eb-a476-d796fb65e81e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bmns9\" (UID: \"42adc88e-a3b3-42eb-a476-d796fb65e81e\") " pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:29.297262 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:29.296752 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42adc88e-a3b3-42eb-a476-d796fb65e81e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bmns9\" (UID: \"42adc88e-a3b3-42eb-a476-d796fb65e81e\") " pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:29.297262 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:29.296817 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 04:54:29.297262 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:29.296830 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 04:54:29.297262 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:29.296884 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42adc88e-a3b3-42eb-a476-d796fb65e81e-kube-rbac-proxy-cm podName:42adc88e-a3b3-42eb-a476-d796fb65e81e nodeName:}" failed. No retries permitted until 2026-07-16 04:54:37.296869074 +0000 UTC m=+267.668992257 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/42adc88e-a3b3-42eb-a476-d796fb65e81e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bmns9" (UID: "42adc88e-a3b3-42eb-a476-d796fb65e81e") : configmap "kube-rbac-proxy" not found Jul 16 04:54:29.297262 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:29.296898 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42adc88e-a3b3-42eb-a476-d796fb65e81e-insights-runtime-extractor-tls podName:42adc88e-a3b3-42eb-a476-d796fb65e81e nodeName:}" failed. No retries permitted until 2026-07-16 04:54:37.296891724 +0000 UTC m=+267.669014906 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/42adc88e-a3b3-42eb-a476-d796fb65e81e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bmns9" (UID: "42adc88e-a3b3-42eb-a476-d796fb65e81e") : secret "insights-runtime-extractor-tls" not found Jul 16 04:54:37.350632 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:37.350594 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42adc88e-a3b3-42eb-a476-d796fb65e81e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bmns9\" (UID: \"42adc88e-a3b3-42eb-a476-d796fb65e81e\") " pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:37.351107 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:37.350641 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42adc88e-a3b3-42eb-a476-d796fb65e81e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bmns9\" (UID: \"42adc88e-a3b3-42eb-a476-d796fb65e81e\") " pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:37.351196 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:37.351171 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42adc88e-a3b3-42eb-a476-d796fb65e81e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bmns9\" (UID: \"42adc88e-a3b3-42eb-a476-d796fb65e81e\") " pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:37.353160 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:37.353139 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42adc88e-a3b3-42eb-a476-d796fb65e81e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bmns9\" (UID: \"42adc88e-a3b3-42eb-a476-d796fb65e81e\") " pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:37.395600 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:37.395569 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bmns9" Jul 16 04:54:37.517428 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:37.517394 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bmns9"] Jul 16 04:54:37.520630 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:54:37.520605 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42adc88e_a3b3_42eb_a476_d796fb65e81e.slice/crio-e7a2c962663ddbd61f217d854f8bc1cb6ae83f418fffcdd5589a9cd336b344cc WatchSource:0}: Error finding container e7a2c962663ddbd61f217d854f8bc1cb6ae83f418fffcdd5589a9cd336b344cc: Status 404 returned error can't find the container with id e7a2c962663ddbd61f217d854f8bc1cb6ae83f418fffcdd5589a9cd336b344cc Jul 16 04:54:37.926993 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:37.926949 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bmns9" event={"ID":"42adc88e-a3b3-42eb-a476-d796fb65e81e","Type":"ContainerStarted","Data":"e7a2c962663ddbd61f217d854f8bc1cb6ae83f418fffcdd5589a9cd336b344cc"} Jul 16 04:54:40.269039 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:40.268989 2561 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" podUID="d4b8d332-7841-435c-b0fb-8a6ddf71aec1" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 16 04:54:42.948610 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:42.948569 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bmns9" event={"ID":"42adc88e-a3b3-42eb-a476-d796fb65e81e","Type":"ContainerStarted","Data":"1ff38106b6446b6bde08e82ab0b42ed07e1c5426a3fa57a9ab801ebb380c405e"} Jul 16 04:54:45.960722 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:45.960665 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bmns9" event={"ID":"42adc88e-a3b3-42eb-a476-d796fb65e81e","Type":"ContainerStarted","Data":"bb9b958a3af5acfd8813179ac4658df2295f111382441450fb3583c0034739ca"} Jul 16 04:54:49.975152 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:49.975115 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bmns9" event={"ID":"42adc88e-a3b3-42eb-a476-d796fb65e81e","Type":"ContainerStarted","Data":"3d518ec7283aeae488c70253517933d5bb1e0e30bde386436f569d45c26dcbe7"} Jul 16 04:54:49.993016 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:49.992973 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bmns9" podStartSLOduration=17.421865003 podStartE2EDuration="28.992960075s" totalImagesPullingTime="11.571095072s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:54:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:54:49.991275453 +0000 UTC m=+280.363398683" watchObservedRunningTime="2026-07-16 04:54:49.992960075 +0000 UTC m=+280.365083280" Jul 16 04:54:50.269303 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:50.269234 2561 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" podUID="d4b8d332-7841-435c-b0fb-8a6ddf71aec1" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 16 04:54:51.630619 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:51.630583 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-q7ghj" podUID="abe48c82-b51a-47de-9586-6fb499424b6a" Jul 16 04:54:51.983389 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:51.983318 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q7ghj" Jul 16 04:54:55.377851 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.377816 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls\") pod \"dns-default-q7ghj\" (UID: \"abe48c82-b51a-47de-9586-6fb499424b6a\") " pod="openshift-dns/dns-default-q7ghj" Jul 16 04:54:55.380157 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.380128 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/abe48c82-b51a-47de-9586-6fb499424b6a-metrics-tls\") pod \"dns-default-q7ghj\" (UID: \"abe48c82-b51a-47de-9586-6fb499424b6a\") " pod="openshift-dns/dns-default-q7ghj" Jul 16 04:54:55.586761 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.586727 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dcm9d\"" Jul 16 04:54:55.594756 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.594737 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q7ghj" Jul 16 04:54:55.693575 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.693549 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-4rxlm"] Jul 16 04:54:55.697549 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.697527 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4rxlm" Jul 16 04:54:55.700572 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.700549 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 16 04:54:55.700785 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.700767 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 16 04:54:55.701786 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.701766 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 16 04:54:55.701887 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.701764 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 16 04:54:55.701887 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.701768 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-gl6p8\"" Jul 16 04:54:55.701887 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.701772 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 16 04:54:55.745754 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.745729 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-4rxlm"] Jul 16 04:54:55.746581 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.746556 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-q7ghj"] Jul 16 04:54:55.748550 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:54:55.748516 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabe48c82_b51a_47de_9586_6fb499424b6a.slice/crio-a9e3fdd650945803ec0ccf5c89e07a4f07446b5bc3f7aabf854198ba4a4ca966 WatchSource:0}: Error finding container a9e3fdd650945803ec0ccf5c89e07a4f07446b5bc3f7aabf854198ba4a4ca966: Status 404 returned error can't find the container with id a9e3fdd650945803ec0ccf5c89e07a4f07446b5bc3f7aabf854198ba4a4ca966 Jul 16 04:54:55.770239 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.770218 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vk7m2"] Jul 16 04:54:55.773303 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.773289 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.775946 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.775928 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 16 04:54:55.776125 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.776112 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m8fw4\"" Jul 16 04:54:55.776279 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.776266 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 16 04:54:55.776348 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.776290 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 16 04:54:55.780600 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.780581 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/877ace0e-1bc5-468e-956d-96b8351dc1cf-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-4rxlm\" (UID: \"877ace0e-1bc5-468e-956d-96b8351dc1cf\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4rxlm" Jul 16 04:54:55.780670 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.780629 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp97j\" (UniqueName: \"kubernetes.io/projected/877ace0e-1bc5-468e-956d-96b8351dc1cf-kube-api-access-hp97j\") pod \"openshift-state-metrics-84bd68b896-4rxlm\" (UID: \"877ace0e-1bc5-468e-956d-96b8351dc1cf\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4rxlm" Jul 16 04:54:55.780670 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.780661 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/877ace0e-1bc5-468e-956d-96b8351dc1cf-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-4rxlm\" (UID: \"877ace0e-1bc5-468e-956d-96b8351dc1cf\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4rxlm" Jul 16 04:54:55.780794 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.780700 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/877ace0e-1bc5-468e-956d-96b8351dc1cf-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-4rxlm\" (UID: \"877ace0e-1bc5-468e-956d-96b8351dc1cf\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4rxlm" Jul 16 04:54:55.881623 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.881590 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/877ace0e-1bc5-468e-956d-96b8351dc1cf-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-4rxlm\" (UID: \"877ace0e-1bc5-468e-956d-96b8351dc1cf\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4rxlm" Jul 16 04:54:55.881814 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.881638 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c2a0834f-b87b-454a-9ce3-75d41b90aead-node-exporter-accelerators-collector-config\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.881814 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.881674 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c2a0834f-b87b-454a-9ce3-75d41b90aead-node-exporter-tls\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.881814 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.881709 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c2a0834f-b87b-454a-9ce3-75d41b90aead-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.881814 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.881741 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c2a0834f-b87b-454a-9ce3-75d41b90aead-root\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.881814 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.881762 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd6f8\" (UniqueName: \"kubernetes.io/projected/c2a0834f-b87b-454a-9ce3-75d41b90aead-kube-api-access-kd6f8\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.881814 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.881780 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2a0834f-b87b-454a-9ce3-75d41b90aead-metrics-client-ca\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.881814 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.881797 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c2a0834f-b87b-454a-9ce3-75d41b90aead-node-exporter-textfile\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.882069 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.881831 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp97j\" (UniqueName: \"kubernetes.io/projected/877ace0e-1bc5-468e-956d-96b8351dc1cf-kube-api-access-hp97j\") pod \"openshift-state-metrics-84bd68b896-4rxlm\" (UID: \"877ace0e-1bc5-468e-956d-96b8351dc1cf\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4rxlm" Jul 16 04:54:55.882069 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.881857 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c2a0834f-b87b-454a-9ce3-75d41b90aead-sys\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.882069 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.881871 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c2a0834f-b87b-454a-9ce3-75d41b90aead-node-exporter-wtmp\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.882069 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.881889 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/877ace0e-1bc5-468e-956d-96b8351dc1cf-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-4rxlm\" (UID: \"877ace0e-1bc5-468e-956d-96b8351dc1cf\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4rxlm" Jul 16 04:54:55.882069 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.881909 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/877ace0e-1bc5-468e-956d-96b8351dc1cf-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-4rxlm\" (UID: \"877ace0e-1bc5-468e-956d-96b8351dc1cf\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4rxlm" Jul 16 04:54:55.882069 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:55.881988 2561 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 16 04:54:55.882069 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:55.882045 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/877ace0e-1bc5-468e-956d-96b8351dc1cf-openshift-state-metrics-tls podName:877ace0e-1bc5-468e-956d-96b8351dc1cf nodeName:}" failed. No retries permitted until 2026-07-16 04:54:56.382025292 +0000 UTC m=+286.754148474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/877ace0e-1bc5-468e-956d-96b8351dc1cf-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-4rxlm" (UID: "877ace0e-1bc5-468e-956d-96b8351dc1cf") : secret "openshift-state-metrics-tls" not found Jul 16 04:54:55.882333 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.882314 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/877ace0e-1bc5-468e-956d-96b8351dc1cf-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-4rxlm\" (UID: \"877ace0e-1bc5-468e-956d-96b8351dc1cf\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4rxlm" Jul 16 04:54:55.884299 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.884266 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/877ace0e-1bc5-468e-956d-96b8351dc1cf-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-4rxlm\" (UID: \"877ace0e-1bc5-468e-956d-96b8351dc1cf\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4rxlm" Jul 16 04:54:55.890289 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.890268 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp97j\" (UniqueName: \"kubernetes.io/projected/877ace0e-1bc5-468e-956d-96b8351dc1cf-kube-api-access-hp97j\") pod \"openshift-state-metrics-84bd68b896-4rxlm\" (UID: \"877ace0e-1bc5-468e-956d-96b8351dc1cf\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4rxlm" Jul 16 04:54:55.983234 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.983147 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c2a0834f-b87b-454a-9ce3-75d41b90aead-root\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.983234 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.983181 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd6f8\" (UniqueName: \"kubernetes.io/projected/c2a0834f-b87b-454a-9ce3-75d41b90aead-kube-api-access-kd6f8\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.983234 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.983201 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2a0834f-b87b-454a-9ce3-75d41b90aead-metrics-client-ca\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.983234 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.983220 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c2a0834f-b87b-454a-9ce3-75d41b90aead-node-exporter-textfile\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.983234 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.983238 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c2a0834f-b87b-454a-9ce3-75d41b90aead-sys\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.983545 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.983254 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c2a0834f-b87b-454a-9ce3-75d41b90aead-node-exporter-wtmp\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.983545 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.983282 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c2a0834f-b87b-454a-9ce3-75d41b90aead-root\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.983545 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.983324 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c2a0834f-b87b-454a-9ce3-75d41b90aead-node-exporter-accelerators-collector-config\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.983545 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.983339 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c2a0834f-b87b-454a-9ce3-75d41b90aead-sys\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.983545 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.983367 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c2a0834f-b87b-454a-9ce3-75d41b90aead-node-exporter-tls\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.983545 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.983392 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c2a0834f-b87b-454a-9ce3-75d41b90aead-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.983545 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.983431 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c2a0834f-b87b-454a-9ce3-75d41b90aead-node-exporter-wtmp\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.983545 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:55.983529 2561 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 16 04:54:55.983877 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:55.983626 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2a0834f-b87b-454a-9ce3-75d41b90aead-node-exporter-tls podName:c2a0834f-b87b-454a-9ce3-75d41b90aead nodeName:}" failed. No retries permitted until 2026-07-16 04:54:56.483609536 +0000 UTC m=+286.855732718 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c2a0834f-b87b-454a-9ce3-75d41b90aead-node-exporter-tls") pod "node-exporter-vk7m2" (UID: "c2a0834f-b87b-454a-9ce3-75d41b90aead") : secret "node-exporter-tls" not found Jul 16 04:54:55.983877 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.983719 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c2a0834f-b87b-454a-9ce3-75d41b90aead-node-exporter-textfile\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.983877 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.983858 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2a0834f-b87b-454a-9ce3-75d41b90aead-metrics-client-ca\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.983976 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.983885 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c2a0834f-b87b-454a-9ce3-75d41b90aead-node-exporter-accelerators-collector-config\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.985738 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.985722 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c2a0834f-b87b-454a-9ce3-75d41b90aead-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.991839 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.991819 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd6f8\" (UniqueName: \"kubernetes.io/projected/c2a0834f-b87b-454a-9ce3-75d41b90aead-kube-api-access-kd6f8\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:55.996062 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:55.995996 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q7ghj" event={"ID":"abe48c82-b51a-47de-9586-6fb499424b6a","Type":"ContainerStarted","Data":"a9e3fdd650945803ec0ccf5c89e07a4f07446b5bc3f7aabf854198ba4a4ca966"} Jul 16 04:54:56.386335 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.386291 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/877ace0e-1bc5-468e-956d-96b8351dc1cf-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-4rxlm\" (UID: \"877ace0e-1bc5-468e-956d-96b8351dc1cf\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4rxlm" Jul 16 04:54:56.389478 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.389448 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/877ace0e-1bc5-468e-956d-96b8351dc1cf-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-4rxlm\" (UID: \"877ace0e-1bc5-468e-956d-96b8351dc1cf\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4rxlm" Jul 16 04:54:56.486831 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.486791 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c2a0834f-b87b-454a-9ce3-75d41b90aead-node-exporter-tls\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:56.489372 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.489346 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c2a0834f-b87b-454a-9ce3-75d41b90aead-node-exporter-tls\") pod \"node-exporter-vk7m2\" (UID: \"c2a0834f-b87b-454a-9ce3-75d41b90aead\") " pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:56.606965 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.606927 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4rxlm" Jul 16 04:54:56.683323 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.683286 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vk7m2" Jul 16 04:54:56.696533 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:54:56.695896 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2a0834f_b87b_454a_9ce3_75d41b90aead.slice/crio-ee831890c2c984dc5b89d1417896fe5470e9d15b1332b19c16b9c01df99c4b81 WatchSource:0}: Error finding container ee831890c2c984dc5b89d1417896fe5470e9d15b1332b19c16b9c01df99c4b81: Status 404 returned error can't find the container with id ee831890c2c984dc5b89d1417896fe5470e9d15b1332b19c16b9c01df99c4b81 Jul 16 04:54:56.749309 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.749285 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-4rxlm"] Jul 16 04:54:56.751989 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:54:56.751946 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod877ace0e_1bc5_468e_956d_96b8351dc1cf.slice/crio-3d498ac32c23aa08e0ef5349db2c6662a205e7c44d427af52d4e072a87bfef8b WatchSource:0}: Error finding container 3d498ac32c23aa08e0ef5349db2c6662a205e7c44d427af52d4e072a87bfef8b: Status 404 returned error can't find the container with id 3d498ac32c23aa08e0ef5349db2c6662a205e7c44d427af52d4e072a87bfef8b Jul 16 04:54:56.779946 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.779923 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 04:54:56.785753 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.785734 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.788339 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.788194 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 16 04:54:56.788339 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.788210 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 16 04:54:56.788339 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.788254 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-s4f9d\"" Jul 16 04:54:56.788339 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.788309 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 16 04:54:56.788652 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.788514 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 16 04:54:56.788652 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.788564 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 16 04:54:56.788652 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.788591 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 16 04:54:56.788827 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.788779 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 16 04:54:56.788883 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.788843 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 16 04:54:56.788932 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.788877 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 16 04:54:56.799143 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.799121 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 04:54:56.889120 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.889089 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.889245 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.889132 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-config-volume\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.889245 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.889162 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1d4fafda-eab9-4056-9281-5dd0af7d8b74-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.889245 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.889228 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1d4fafda-eab9-4056-9281-5dd0af7d8b74-config-out\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.889245 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.889247 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-web-config\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.889428 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.889261 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.889428 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.889281 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1d4fafda-eab9-4056-9281-5dd0af7d8b74-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.889428 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.889336 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d4fafda-eab9-4056-9281-5dd0af7d8b74-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.889428 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.889395 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdw49\" (UniqueName: \"kubernetes.io/projected/1d4fafda-eab9-4056-9281-5dd0af7d8b74-kube-api-access-cdw49\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.889610 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.889432 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.889610 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.889496 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.889610 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.889524 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1d4fafda-eab9-4056-9281-5dd0af7d8b74-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.889610 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.889576 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.990858 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.990735 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1d4fafda-eab9-4056-9281-5dd0af7d8b74-config-out\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.990858 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.990789 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-web-config\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.990858 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.990809 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.990858 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.990835 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1d4fafda-eab9-4056-9281-5dd0af7d8b74-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.991171 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.990877 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d4fafda-eab9-4056-9281-5dd0af7d8b74-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.991171 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.990912 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdw49\" (UniqueName: \"kubernetes.io/projected/1d4fafda-eab9-4056-9281-5dd0af7d8b74-kube-api-access-cdw49\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.991171 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.990928 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.991171 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.990966 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.991171 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.990993 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1d4fafda-eab9-4056-9281-5dd0af7d8b74-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.991171 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.991015 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.991171 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.991045 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.991171 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:56.991088 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1d4fafda-eab9-4056-9281-5dd0af7d8b74-alertmanager-trusted-ca-bundle podName:1d4fafda-eab9-4056-9281-5dd0af7d8b74 nodeName:}" failed. No retries permitted until 2026-07-16 04:54:57.49104577 +0000 UTC m=+287.863168966 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/1d4fafda-eab9-4056-9281-5dd0af7d8b74-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "1d4fafda-eab9-4056-9281-5dd0af7d8b74") : configmap references non-existent config key: ca-bundle.crt Jul 16 04:54:56.991171 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.991135 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-config-volume\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.991171 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.991166 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1d4fafda-eab9-4056-9281-5dd0af7d8b74-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.991710 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:56.991673 2561 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 16 04:54:56.991815 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.991729 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1d4fafda-eab9-4056-9281-5dd0af7d8b74-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.991815 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:54:56.991753 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-main-tls podName:1d4fafda-eab9-4056-9281-5dd0af7d8b74 nodeName:}" failed. No retries permitted until 2026-07-16 04:54:57.491735587 +0000 UTC m=+287.863858771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "1d4fafda-eab9-4056-9281-5dd0af7d8b74") : secret "alertmanager-main-tls" not found Jul 16 04:54:56.994808 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.994412 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1d4fafda-eab9-4056-9281-5dd0af7d8b74-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.996804 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.995406 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1d4fafda-eab9-4056-9281-5dd0af7d8b74-config-out\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.996804 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.996022 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.997014 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.996842 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.999975 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.999548 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-config-volume\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.999975 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.999615 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:56.999975 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.999670 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdw49\" (UniqueName: \"kubernetes.io/projected/1d4fafda-eab9-4056-9281-5dd0af7d8b74-kube-api-access-cdw49\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:57.000163 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:56.999979 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1d4fafda-eab9-4056-9281-5dd0af7d8b74-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:57.000163 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:57.000034 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-web-config\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:57.001173 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:57.001138 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:57.004259 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:57.004228 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4rxlm" event={"ID":"877ace0e-1bc5-468e-956d-96b8351dc1cf","Type":"ContainerStarted","Data":"27ae95e1131f44e50a93e88267b6fe72f46bcaa25ff742cac4d135b8706f80d0"} Jul 16 04:54:57.004375 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:57.004332 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4rxlm" event={"ID":"877ace0e-1bc5-468e-956d-96b8351dc1cf","Type":"ContainerStarted","Data":"87b36084450178e0b9fce1b801f0fa90560487b2f17c52cca93477df302e1365"} Jul 16 04:54:57.004375 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:57.004362 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4rxlm" event={"ID":"877ace0e-1bc5-468e-956d-96b8351dc1cf","Type":"ContainerStarted","Data":"3d498ac32c23aa08e0ef5349db2c6662a205e7c44d427af52d4e072a87bfef8b"} Jul 16 04:54:57.005509 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:57.005485 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vk7m2" event={"ID":"c2a0834f-b87b-454a-9ce3-75d41b90aead","Type":"ContainerStarted","Data":"ee831890c2c984dc5b89d1417896fe5470e9d15b1332b19c16b9c01df99c4b81"} Jul 16 04:54:57.496808 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:57.496661 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:57.496808 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:57.496799 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d4fafda-eab9-4056-9281-5dd0af7d8b74-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:57.497734 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:57.497706 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d4fafda-eab9-4056-9281-5dd0af7d8b74-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:57.499786 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:57.499764 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:57.696851 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:57.696665 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:54:58.013297 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:58.012806 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q7ghj" event={"ID":"abe48c82-b51a-47de-9586-6fb499424b6a","Type":"ContainerStarted","Data":"5fa80300920c457ba0f1f8f81b183b646f0fec0ef6b04f27b817aad4535296d0"} Jul 16 04:54:58.013297 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:58.012868 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q7ghj" event={"ID":"abe48c82-b51a-47de-9586-6fb499424b6a","Type":"ContainerStarted","Data":"74aa3e3be75b0f895ab5482954092b29e6a574ae40a9fc53d73d0222db386b22"} Jul 16 04:54:58.014188 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:58.013453 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-q7ghj" Jul 16 04:54:58.029964 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:58.029860 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-q7ghj" podStartSLOduration=251.357014591 podStartE2EDuration="4m13.029840432s" totalImagesPullingTime="1.672825841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:50:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:54:58.028661952 +0000 UTC m=+288.400785183" watchObservedRunningTime="2026-07-16 04:54:58.029840432 +0000 UTC m=+288.401963639" Jul 16 04:54:58.062245 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:58.062220 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 04:54:58.253482 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:54:58.253408 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d4fafda_eab9_4056_9281_5dd0af7d8b74.slice/crio-1a9e2cef22a366f3c2c6d188611ec9ef8c62296d54cf1e89232112c6a5798d7c WatchSource:0}: Error finding container 1a9e2cef22a366f3c2c6d188611ec9ef8c62296d54cf1e89232112c6a5798d7c: Status 404 returned error can't find the container with id 1a9e2cef22a366f3c2c6d188611ec9ef8c62296d54cf1e89232112c6a5798d7c Jul 16 04:54:59.020442 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:59.020403 2561 generic.go:356] "Generic (PLEG): container finished" podID="c2a0834f-b87b-454a-9ce3-75d41b90aead" containerID="897f7d789c8da511d9281d50cc20d3ffc0a1a5276ca40562b4bf1d55451e63e7" exitCode=0 Jul 16 04:54:59.020909 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:59.020488 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vk7m2" event={"ID":"c2a0834f-b87b-454a-9ce3-75d41b90aead","Type":"ContainerDied","Data":"897f7d789c8da511d9281d50cc20d3ffc0a1a5276ca40562b4bf1d55451e63e7"} Jul 16 04:54:59.022712 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:59.022671 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4rxlm" event={"ID":"877ace0e-1bc5-468e-956d-96b8351dc1cf","Type":"ContainerStarted","Data":"58b5458cf47f15255ec1013c720e7e187b6e7a63f22747d77a594d87eb136781"} Jul 16 04:54:59.023924 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:59.023906 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1d4fafda-eab9-4056-9281-5dd0af7d8b74","Type":"ContainerStarted","Data":"1a9e2cef22a366f3c2c6d188611ec9ef8c62296d54cf1e89232112c6a5798d7c"} Jul 16 04:54:59.053798 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:54:59.053755 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4rxlm" podStartSLOduration=2.6449104180000003 podStartE2EDuration="4.053739474s" totalImagesPullingTime="1.408829056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:54:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:54:59.053589583 +0000 UTC m=+289.425712789" watchObservedRunningTime="2026-07-16 04:54:59.053739474 +0000 UTC m=+289.425862660" Jul 16 04:55:00.029115 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:00.029076 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vk7m2" event={"ID":"c2a0834f-b87b-454a-9ce3-75d41b90aead","Type":"ContainerStarted","Data":"2faf9998ddf1d8128d453a25e3fc0dffee548f5789be46ebc83390fa87d8ee59"} Jul 16 04:55:00.029569 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:00.029136 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vk7m2" event={"ID":"c2a0834f-b87b-454a-9ce3-75d41b90aead","Type":"ContainerStarted","Data":"f14e22da2bfa3a063ec13602128ec3a35f3f5e10a426382deb381cc309921eab"} Jul 16 04:55:00.051056 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:00.051014 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vk7m2" podStartSLOduration=3.704497092 podStartE2EDuration="5.050998689s" totalImagesPullingTime="1.268961763s" totalInitContainerRuntime="77.539834ms" isStatefulPod=true podCreationTimestamp="2026-07-16 04:54:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:55:00.049138353 +0000 UTC m=+290.421261579" watchObservedRunningTime="2026-07-16 04:55:00.050998689 +0000 UTC m=+290.423121893" Jul 16 04:55:00.268761 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:00.268719 2561 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" podUID="d4b8d332-7841-435c-b0fb-8a6ddf71aec1" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 16 04:55:00.269019 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:00.268833 2561 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" Jul 16 04:55:00.269400 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:00.269366 2561 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"b1ea6a20da8d88fb945f05dae6aae7daa4a62fcc82777fc526c2f6f2505d9c3c"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 16 04:55:00.269494 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:00.269423 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" podUID="d4b8d332-7841-435c-b0fb-8a6ddf71aec1" containerName="service-proxy" containerID="cri-o://b1ea6a20da8d88fb945f05dae6aae7daa4a62fcc82777fc526c2f6f2505d9c3c" gracePeriod=30 Jul 16 04:55:01.033430 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:01.033346 2561 generic.go:356] "Generic (PLEG): container finished" podID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerID="a6ced93a8bf6f67c4966b02208ee257c825c3a9c22eb3d9af6bf4a1a881999c0" exitCode=0 Jul 16 04:55:01.033893 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:01.033444 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1d4fafda-eab9-4056-9281-5dd0af7d8b74","Type":"ContainerDied","Data":"a6ced93a8bf6f67c4966b02208ee257c825c3a9c22eb3d9af6bf4a1a881999c0"} Jul 16 04:55:01.035915 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:01.035892 2561 generic.go:356] "Generic (PLEG): container finished" podID="d4b8d332-7841-435c-b0fb-8a6ddf71aec1" containerID="b1ea6a20da8d88fb945f05dae6aae7daa4a62fcc82777fc526c2f6f2505d9c3c" exitCode=2 Jul 16 04:55:01.036024 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:01.035973 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" event={"ID":"d4b8d332-7841-435c-b0fb-8a6ddf71aec1","Type":"ContainerDied","Data":"b1ea6a20da8d88fb945f05dae6aae7daa4a62fcc82777fc526c2f6f2505d9c3c"} Jul 16 04:55:01.036024 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:01.036012 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f577b6848-wxngm" event={"ID":"d4b8d332-7841-435c-b0fb-8a6ddf71aec1","Type":"ContainerStarted","Data":"f17cbe1715fb803e0b2d005bf533cce69f3718c8b6e1b5d9d89dc91113f95f65"} Jul 16 04:55:03.046309 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:03.046286 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1d4fafda-eab9-4056-9281-5dd0af7d8b74","Type":"ContainerStarted","Data":"33150fa1288ab8415dc6b6e27780a7479a2e7f9ad64f12e56355614bb3d5dca6"} Jul 16 04:55:03.046595 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:03.046351 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1d4fafda-eab9-4056-9281-5dd0af7d8b74","Type":"ContainerStarted","Data":"03544c882793aa17807f36729eee45c6321182ebd9c84dcb5756a14d7e41a328"} Jul 16 04:55:03.046595 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:03.046377 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1d4fafda-eab9-4056-9281-5dd0af7d8b74","Type":"ContainerStarted","Data":"15ada47c76554c150a3be6206a14e468731e4a8afa12b018706a632b4c204860"} Jul 16 04:55:03.097100 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:55:03.097044 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-2srzm" podUID="8b4ec3ce-70fc-4716-a3c6-5ab123a8be45" Jul 16 04:55:04.053083 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:04.053050 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1d4fafda-eab9-4056-9281-5dd0af7d8b74","Type":"ContainerStarted","Data":"d071fa45d642239840b0ff1088e1cfcb5c63f1ff2536e195d56650332a2abfa2"} Jul 16 04:55:04.053434 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:04.053121 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1d4fafda-eab9-4056-9281-5dd0af7d8b74","Type":"ContainerStarted","Data":"6161c77ecb07b6cc8f0670710fce6ddc5de0a83aa1915629d528bb6c343fe445"} Jul 16 04:55:04.053434 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:04.053146 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1d4fafda-eab9-4056-9281-5dd0af7d8b74","Type":"ContainerStarted","Data":"7f211a9c360cca8c0526ed43298e8465426defbc7ceef103fd8728215bff5ac5"} Jul 16 04:55:04.079080 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:04.079026 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.965689739 podStartE2EDuration="8.079010556s" totalImagesPullingTime="5.099430089s" totalInitContainerRuntime="13.890728ms" isStatefulPod=false podCreationTimestamp="2026-07-16 04:54:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:55:04.077303175 +0000 UTC m=+294.449426396" watchObservedRunningTime="2026-07-16 04:55:04.079010556 +0000 UTC m=+294.451133742" Jul 16 04:55:08.026190 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:08.026162 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-q7ghj" Jul 16 04:55:09.997171 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:09.997147 2561 kubelet.go:1648] "Image garbage collection succeeded" Jul 16 04:55:16.094922 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:16.094887 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:55:19.268449 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.268407 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-54bcd668c4-mvt6j"] Jul 16 04:55:19.445711 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.445649 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54bcd668c4-mvt6j"] Jul 16 04:55:19.445711 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.445715 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:19.448836 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.448803 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 16 04:55:19.448966 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.448844 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 16 04:55:19.448966 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.448856 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 16 04:55:19.449960 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.449933 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4bwwt\"" Jul 16 04:55:19.450117 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.450007 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 16 04:55:19.450117 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.450026 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 16 04:55:19.450117 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.450039 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 16 04:55:19.450334 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.450251 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 16 04:55:19.454712 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.454581 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 16 04:55:19.574483 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.574452 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4n2z\" (UniqueName: \"kubernetes.io/projected/b5c61c5b-7313-42aa-a0ec-1340c883b107-kube-api-access-x4n2z\") pod \"console-54bcd668c4-mvt6j\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:19.574649 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.574494 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b5c61c5b-7313-42aa-a0ec-1340c883b107-console-serving-cert\") pod \"console-54bcd668c4-mvt6j\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:19.574649 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.574517 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b5c61c5b-7313-42aa-a0ec-1340c883b107-console-config\") pod \"console-54bcd668c4-mvt6j\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:19.574649 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.574534 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b5c61c5b-7313-42aa-a0ec-1340c883b107-oauth-serving-cert\") pod \"console-54bcd668c4-mvt6j\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:19.574649 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.574634 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b5c61c5b-7313-42aa-a0ec-1340c883b107-service-ca\") pod \"console-54bcd668c4-mvt6j\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:19.574847 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.574670 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b5c61c5b-7313-42aa-a0ec-1340c883b107-console-oauth-config\") pod \"console-54bcd668c4-mvt6j\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:19.574847 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.574728 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5c61c5b-7313-42aa-a0ec-1340c883b107-trusted-ca-bundle\") pod \"console-54bcd668c4-mvt6j\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:19.675966 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.675926 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4n2z\" (UniqueName: \"kubernetes.io/projected/b5c61c5b-7313-42aa-a0ec-1340c883b107-kube-api-access-x4n2z\") pod \"console-54bcd668c4-mvt6j\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:19.675966 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.675968 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b5c61c5b-7313-42aa-a0ec-1340c883b107-console-serving-cert\") pod \"console-54bcd668c4-mvt6j\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:19.676179 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.675994 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b5c61c5b-7313-42aa-a0ec-1340c883b107-console-config\") pod \"console-54bcd668c4-mvt6j\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:19.676179 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.676018 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b5c61c5b-7313-42aa-a0ec-1340c883b107-oauth-serving-cert\") pod \"console-54bcd668c4-mvt6j\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:19.676179 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.676065 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b5c61c5b-7313-42aa-a0ec-1340c883b107-service-ca\") pod \"console-54bcd668c4-mvt6j\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:19.676179 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.676091 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b5c61c5b-7313-42aa-a0ec-1340c883b107-console-oauth-config\") pod \"console-54bcd668c4-mvt6j\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:19.676179 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.676127 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5c61c5b-7313-42aa-a0ec-1340c883b107-trusted-ca-bundle\") pod \"console-54bcd668c4-mvt6j\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:19.676783 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.676759 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b5c61c5b-7313-42aa-a0ec-1340c883b107-console-config\") pod \"console-54bcd668c4-mvt6j\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:19.676925 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.676901 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b5c61c5b-7313-42aa-a0ec-1340c883b107-service-ca\") pod \"console-54bcd668c4-mvt6j\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:19.677040 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.677025 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5c61c5b-7313-42aa-a0ec-1340c883b107-trusted-ca-bundle\") pod \"console-54bcd668c4-mvt6j\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:19.677089 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.677048 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b5c61c5b-7313-42aa-a0ec-1340c883b107-oauth-serving-cert\") pod \"console-54bcd668c4-mvt6j\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:19.678570 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.678548 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b5c61c5b-7313-42aa-a0ec-1340c883b107-console-serving-cert\") pod \"console-54bcd668c4-mvt6j\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:19.678677 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.678605 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b5c61c5b-7313-42aa-a0ec-1340c883b107-console-oauth-config\") pod \"console-54bcd668c4-mvt6j\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:19.684040 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.684017 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4n2z\" (UniqueName: \"kubernetes.io/projected/b5c61c5b-7313-42aa-a0ec-1340c883b107-kube-api-access-x4n2z\") pod \"console-54bcd668c4-mvt6j\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:19.755986 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.755946 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:19.943873 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.943849 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54bcd668c4-mvt6j"] Jul 16 04:55:19.946482 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:55:19.946445 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5c61c5b_7313_42aa_a0ec_1340c883b107.slice/crio-0efa07ee98742b0dd349efbae6af38ea72a9301a2a1b2e193156876b80207a36 WatchSource:0}: Error finding container 0efa07ee98742b0dd349efbae6af38ea72a9301a2a1b2e193156876b80207a36: Status 404 returned error can't find the container with id 0efa07ee98742b0dd349efbae6af38ea72a9301a2a1b2e193156876b80207a36 Jul 16 04:55:19.948392 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:19.948377 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 04:55:20.110545 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:20.110463 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54bcd668c4-mvt6j" event={"ID":"b5c61c5b-7313-42aa-a0ec-1340c883b107","Type":"ContainerStarted","Data":"0efa07ee98742b0dd349efbae6af38ea72a9301a2a1b2e193156876b80207a36"} Jul 16 04:55:23.124248 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:23.124215 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54bcd668c4-mvt6j" event={"ID":"b5c61c5b-7313-42aa-a0ec-1340c883b107","Type":"ContainerStarted","Data":"e56655060a962adf8c5e660084bc46039926f68dd04d1fedb9a3c3c617e579c6"} Jul 16 04:55:23.145480 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:23.142294 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-54bcd668c4-mvt6j" podStartSLOduration=1.4939836770000001 podStartE2EDuration="4.142268792s" totalImagesPullingTime="2.648285115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:55:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:55:23.139764727 +0000 UTC m=+313.511887960" watchObservedRunningTime="2026-07-16 04:55:23.142268792 +0000 UTC m=+313.514391998" Jul 16 04:55:29.756702 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:29.756643 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:29.757177 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:29.756723 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:29.761491 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:29.761463 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:55:30.151907 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:55:30.151879 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:56:16.032706 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:16.032654 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 04:56:16.033301 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:16.033041 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="alertmanager" containerID="cri-o://15ada47c76554c150a3be6206a14e468731e4a8afa12b018706a632b4c204860" gracePeriod=120 Jul 16 04:56:16.033301 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:16.033098 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="kube-rbac-proxy-metric" containerID="cri-o://6161c77ecb07b6cc8f0670710fce6ddc5de0a83aa1915629d528bb6c343fe445" gracePeriod=120 Jul 16 04:56:16.033301 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:16.033126 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="prom-label-proxy" containerID="cri-o://d071fa45d642239840b0ff1088e1cfcb5c63f1ff2536e195d56650332a2abfa2" gracePeriod=120 Jul 16 04:56:16.033301 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:16.033168 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="config-reloader" containerID="cri-o://03544c882793aa17807f36729eee45c6321182ebd9c84dcb5756a14d7e41a328" gracePeriod=120 Jul 16 04:56:16.033301 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:16.033195 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="kube-rbac-proxy" containerID="cri-o://7f211a9c360cca8c0526ed43298e8465426defbc7ceef103fd8728215bff5ac5" gracePeriod=120 Jul 16 04:56:16.033301 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:16.033155 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="kube-rbac-proxy-web" containerID="cri-o://33150fa1288ab8415dc6b6e27780a7479a2e7f9ad64f12e56355614bb3d5dca6" gracePeriod=120 Jul 16 04:56:16.310621 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:16.310595 2561 generic.go:356] "Generic (PLEG): container finished" podID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerID="d071fa45d642239840b0ff1088e1cfcb5c63f1ff2536e195d56650332a2abfa2" exitCode=0 Jul 16 04:56:16.310621 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:16.310617 2561 generic.go:356] "Generic (PLEG): container finished" podID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerID="7f211a9c360cca8c0526ed43298e8465426defbc7ceef103fd8728215bff5ac5" exitCode=0 Jul 16 04:56:16.310621 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:16.310624 2561 generic.go:356] "Generic (PLEG): container finished" podID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerID="03544c882793aa17807f36729eee45c6321182ebd9c84dcb5756a14d7e41a328" exitCode=0 Jul 16 04:56:16.310621 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:16.310629 2561 generic.go:356] "Generic (PLEG): container finished" podID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerID="15ada47c76554c150a3be6206a14e468731e4a8afa12b018706a632b4c204860" exitCode=0 Jul 16 04:56:16.310891 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:16.310669 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1d4fafda-eab9-4056-9281-5dd0af7d8b74","Type":"ContainerDied","Data":"d071fa45d642239840b0ff1088e1cfcb5c63f1ff2536e195d56650332a2abfa2"} Jul 16 04:56:16.310891 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:16.310763 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1d4fafda-eab9-4056-9281-5dd0af7d8b74","Type":"ContainerDied","Data":"7f211a9c360cca8c0526ed43298e8465426defbc7ceef103fd8728215bff5ac5"} Jul 16 04:56:16.310891 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:16.310790 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1d4fafda-eab9-4056-9281-5dd0af7d8b74","Type":"ContainerDied","Data":"03544c882793aa17807f36729eee45c6321182ebd9c84dcb5756a14d7e41a328"} Jul 16 04:56:16.310891 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:16.310814 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1d4fafda-eab9-4056-9281-5dd0af7d8b74","Type":"ContainerDied","Data":"15ada47c76554c150a3be6206a14e468731e4a8afa12b018706a632b4c204860"} Jul 16 04:56:17.268755 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.268733 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.317115 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.317086 2561 generic.go:356] "Generic (PLEG): container finished" podID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerID="6161c77ecb07b6cc8f0670710fce6ddc5de0a83aa1915629d528bb6c343fe445" exitCode=0 Jul 16 04:56:17.317115 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.317113 2561 generic.go:356] "Generic (PLEG): container finished" podID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerID="33150fa1288ab8415dc6b6e27780a7479a2e7f9ad64f12e56355614bb3d5dca6" exitCode=0 Jul 16 04:56:17.317301 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.317176 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1d4fafda-eab9-4056-9281-5dd0af7d8b74","Type":"ContainerDied","Data":"6161c77ecb07b6cc8f0670710fce6ddc5de0a83aa1915629d528bb6c343fe445"} Jul 16 04:56:17.317301 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.317247 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.317301 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.317269 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1d4fafda-eab9-4056-9281-5dd0af7d8b74","Type":"ContainerDied","Data":"33150fa1288ab8415dc6b6e27780a7479a2e7f9ad64f12e56355614bb3d5dca6"} Jul 16 04:56:17.317301 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.317299 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1d4fafda-eab9-4056-9281-5dd0af7d8b74","Type":"ContainerDied","Data":"1a9e2cef22a366f3c2c6d188611ec9ef8c62296d54cf1e89232112c6a5798d7c"} Jul 16 04:56:17.317430 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.317340 2561 scope.go:117] "RemoveContainer" containerID="d071fa45d642239840b0ff1088e1cfcb5c63f1ff2536e195d56650332a2abfa2" Jul 16 04:56:17.325432 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.325411 2561 scope.go:117] "RemoveContainer" containerID="6161c77ecb07b6cc8f0670710fce6ddc5de0a83aa1915629d528bb6c343fe445" Jul 16 04:56:17.332256 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.332240 2561 scope.go:117] "RemoveContainer" containerID="7f211a9c360cca8c0526ed43298e8465426defbc7ceef103fd8728215bff5ac5" Jul 16 04:56:17.338816 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.338794 2561 scope.go:117] "RemoveContainer" containerID="33150fa1288ab8415dc6b6e27780a7479a2e7f9ad64f12e56355614bb3d5dca6" Jul 16 04:56:17.345415 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.345378 2561 scope.go:117] "RemoveContainer" containerID="03544c882793aa17807f36729eee45c6321182ebd9c84dcb5756a14d7e41a328" Jul 16 04:56:17.352182 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.352167 2561 scope.go:117] "RemoveContainer" containerID="15ada47c76554c150a3be6206a14e468731e4a8afa12b018706a632b4c204860" Jul 16 04:56:17.358784 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.358764 2561 scope.go:117] "RemoveContainer" containerID="a6ced93a8bf6f67c4966b02208ee257c825c3a9c22eb3d9af6bf4a1a881999c0" Jul 16 04:56:17.364909 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.364884 2561 scope.go:117] "RemoveContainer" containerID="d071fa45d642239840b0ff1088e1cfcb5c63f1ff2536e195d56650332a2abfa2" Jul 16 04:56:17.365169 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:56:17.365145 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d071fa45d642239840b0ff1088e1cfcb5c63f1ff2536e195d56650332a2abfa2\": container with ID starting with d071fa45d642239840b0ff1088e1cfcb5c63f1ff2536e195d56650332a2abfa2 not found: ID does not exist" containerID="d071fa45d642239840b0ff1088e1cfcb5c63f1ff2536e195d56650332a2abfa2" Jul 16 04:56:17.365253 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.365185 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d071fa45d642239840b0ff1088e1cfcb5c63f1ff2536e195d56650332a2abfa2"} err="failed to get container status \"d071fa45d642239840b0ff1088e1cfcb5c63f1ff2536e195d56650332a2abfa2\": rpc error: code = NotFound desc = could not find container \"d071fa45d642239840b0ff1088e1cfcb5c63f1ff2536e195d56650332a2abfa2\": container with ID starting with d071fa45d642239840b0ff1088e1cfcb5c63f1ff2536e195d56650332a2abfa2 not found: ID does not exist" Jul 16 04:56:17.365253 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.365212 2561 scope.go:117] "RemoveContainer" containerID="6161c77ecb07b6cc8f0670710fce6ddc5de0a83aa1915629d528bb6c343fe445" Jul 16 04:56:17.365462 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:56:17.365446 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6161c77ecb07b6cc8f0670710fce6ddc5de0a83aa1915629d528bb6c343fe445\": container with ID starting with 6161c77ecb07b6cc8f0670710fce6ddc5de0a83aa1915629d528bb6c343fe445 not found: ID does not exist" containerID="6161c77ecb07b6cc8f0670710fce6ddc5de0a83aa1915629d528bb6c343fe445" Jul 16 04:56:17.365503 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.365468 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6161c77ecb07b6cc8f0670710fce6ddc5de0a83aa1915629d528bb6c343fe445"} err="failed to get container status \"6161c77ecb07b6cc8f0670710fce6ddc5de0a83aa1915629d528bb6c343fe445\": rpc error: code = NotFound desc = could not find container \"6161c77ecb07b6cc8f0670710fce6ddc5de0a83aa1915629d528bb6c343fe445\": container with ID starting with 6161c77ecb07b6cc8f0670710fce6ddc5de0a83aa1915629d528bb6c343fe445 not found: ID does not exist" Jul 16 04:56:17.365503 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.365484 2561 scope.go:117] "RemoveContainer" containerID="7f211a9c360cca8c0526ed43298e8465426defbc7ceef103fd8728215bff5ac5" Jul 16 04:56:17.365721 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:56:17.365705 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f211a9c360cca8c0526ed43298e8465426defbc7ceef103fd8728215bff5ac5\": container with ID starting with 7f211a9c360cca8c0526ed43298e8465426defbc7ceef103fd8728215bff5ac5 not found: ID does not exist" containerID="7f211a9c360cca8c0526ed43298e8465426defbc7ceef103fd8728215bff5ac5" Jul 16 04:56:17.365789 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.365729 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f211a9c360cca8c0526ed43298e8465426defbc7ceef103fd8728215bff5ac5"} err="failed to get container status \"7f211a9c360cca8c0526ed43298e8465426defbc7ceef103fd8728215bff5ac5\": rpc error: code = NotFound desc = could not find container \"7f211a9c360cca8c0526ed43298e8465426defbc7ceef103fd8728215bff5ac5\": container with ID starting with 7f211a9c360cca8c0526ed43298e8465426defbc7ceef103fd8728215bff5ac5 not found: ID does not exist" Jul 16 04:56:17.365789 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.365750 2561 scope.go:117] "RemoveContainer" containerID="33150fa1288ab8415dc6b6e27780a7479a2e7f9ad64f12e56355614bb3d5dca6" Jul 16 04:56:17.365981 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:56:17.365960 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33150fa1288ab8415dc6b6e27780a7479a2e7f9ad64f12e56355614bb3d5dca6\": container with ID starting with 33150fa1288ab8415dc6b6e27780a7479a2e7f9ad64f12e56355614bb3d5dca6 not found: ID does not exist" containerID="33150fa1288ab8415dc6b6e27780a7479a2e7f9ad64f12e56355614bb3d5dca6" Jul 16 04:56:17.366029 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.365986 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33150fa1288ab8415dc6b6e27780a7479a2e7f9ad64f12e56355614bb3d5dca6"} err="failed to get container status \"33150fa1288ab8415dc6b6e27780a7479a2e7f9ad64f12e56355614bb3d5dca6\": rpc error: code = NotFound desc = could not find container \"33150fa1288ab8415dc6b6e27780a7479a2e7f9ad64f12e56355614bb3d5dca6\": container with ID starting with 33150fa1288ab8415dc6b6e27780a7479a2e7f9ad64f12e56355614bb3d5dca6 not found: ID does not exist" Jul 16 04:56:17.366029 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.366000 2561 scope.go:117] "RemoveContainer" containerID="03544c882793aa17807f36729eee45c6321182ebd9c84dcb5756a14d7e41a328" Jul 16 04:56:17.366195 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:56:17.366177 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03544c882793aa17807f36729eee45c6321182ebd9c84dcb5756a14d7e41a328\": container with ID starting with 03544c882793aa17807f36729eee45c6321182ebd9c84dcb5756a14d7e41a328 not found: ID does not exist" containerID="03544c882793aa17807f36729eee45c6321182ebd9c84dcb5756a14d7e41a328" Jul 16 04:56:17.366237 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.366202 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03544c882793aa17807f36729eee45c6321182ebd9c84dcb5756a14d7e41a328"} err="failed to get container status \"03544c882793aa17807f36729eee45c6321182ebd9c84dcb5756a14d7e41a328\": rpc error: code = NotFound desc = could not find container \"03544c882793aa17807f36729eee45c6321182ebd9c84dcb5756a14d7e41a328\": container with ID starting with 03544c882793aa17807f36729eee45c6321182ebd9c84dcb5756a14d7e41a328 not found: ID does not exist" Jul 16 04:56:17.366237 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.366216 2561 scope.go:117] "RemoveContainer" containerID="15ada47c76554c150a3be6206a14e468731e4a8afa12b018706a632b4c204860" Jul 16 04:56:17.366406 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:56:17.366387 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15ada47c76554c150a3be6206a14e468731e4a8afa12b018706a632b4c204860\": container with ID starting with 15ada47c76554c150a3be6206a14e468731e4a8afa12b018706a632b4c204860 not found: ID does not exist" containerID="15ada47c76554c150a3be6206a14e468731e4a8afa12b018706a632b4c204860" Jul 16 04:56:17.366446 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.366410 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15ada47c76554c150a3be6206a14e468731e4a8afa12b018706a632b4c204860"} err="failed to get container status \"15ada47c76554c150a3be6206a14e468731e4a8afa12b018706a632b4c204860\": rpc error: code = NotFound desc = could not find container \"15ada47c76554c150a3be6206a14e468731e4a8afa12b018706a632b4c204860\": container with ID starting with 15ada47c76554c150a3be6206a14e468731e4a8afa12b018706a632b4c204860 not found: ID does not exist" Jul 16 04:56:17.366446 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.366421 2561 scope.go:117] "RemoveContainer" containerID="a6ced93a8bf6f67c4966b02208ee257c825c3a9c22eb3d9af6bf4a1a881999c0" Jul 16 04:56:17.366611 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:56:17.366596 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6ced93a8bf6f67c4966b02208ee257c825c3a9c22eb3d9af6bf4a1a881999c0\": container with ID starting with a6ced93a8bf6f67c4966b02208ee257c825c3a9c22eb3d9af6bf4a1a881999c0 not found: ID does not exist" containerID="a6ced93a8bf6f67c4966b02208ee257c825c3a9c22eb3d9af6bf4a1a881999c0" Jul 16 04:56:17.366661 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.366615 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6ced93a8bf6f67c4966b02208ee257c825c3a9c22eb3d9af6bf4a1a881999c0"} err="failed to get container status \"a6ced93a8bf6f67c4966b02208ee257c825c3a9c22eb3d9af6bf4a1a881999c0\": rpc error: code = NotFound desc = could not find container \"a6ced93a8bf6f67c4966b02208ee257c825c3a9c22eb3d9af6bf4a1a881999c0\": container with ID starting with a6ced93a8bf6f67c4966b02208ee257c825c3a9c22eb3d9af6bf4a1a881999c0 not found: ID does not exist" Jul 16 04:56:17.366661 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.366626 2561 scope.go:117] "RemoveContainer" containerID="d071fa45d642239840b0ff1088e1cfcb5c63f1ff2536e195d56650332a2abfa2" Jul 16 04:56:17.366847 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.366832 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d071fa45d642239840b0ff1088e1cfcb5c63f1ff2536e195d56650332a2abfa2"} err="failed to get container status \"d071fa45d642239840b0ff1088e1cfcb5c63f1ff2536e195d56650332a2abfa2\": rpc error: code = NotFound desc = could not find container \"d071fa45d642239840b0ff1088e1cfcb5c63f1ff2536e195d56650332a2abfa2\": container with ID starting with d071fa45d642239840b0ff1088e1cfcb5c63f1ff2536e195d56650332a2abfa2 not found: ID does not exist" Jul 16 04:56:17.366888 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.366849 2561 scope.go:117] "RemoveContainer" containerID="6161c77ecb07b6cc8f0670710fce6ddc5de0a83aa1915629d528bb6c343fe445" Jul 16 04:56:17.367050 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.367033 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6161c77ecb07b6cc8f0670710fce6ddc5de0a83aa1915629d528bb6c343fe445"} err="failed to get container status \"6161c77ecb07b6cc8f0670710fce6ddc5de0a83aa1915629d528bb6c343fe445\": rpc error: code = NotFound desc = could not find container \"6161c77ecb07b6cc8f0670710fce6ddc5de0a83aa1915629d528bb6c343fe445\": container with ID starting with 6161c77ecb07b6cc8f0670710fce6ddc5de0a83aa1915629d528bb6c343fe445 not found: ID does not exist" Jul 16 04:56:17.367109 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.367051 2561 scope.go:117] "RemoveContainer" containerID="7f211a9c360cca8c0526ed43298e8465426defbc7ceef103fd8728215bff5ac5" Jul 16 04:56:17.367239 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.367224 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f211a9c360cca8c0526ed43298e8465426defbc7ceef103fd8728215bff5ac5"} err="failed to get container status \"7f211a9c360cca8c0526ed43298e8465426defbc7ceef103fd8728215bff5ac5\": rpc error: code = NotFound desc = could not find container \"7f211a9c360cca8c0526ed43298e8465426defbc7ceef103fd8728215bff5ac5\": container with ID starting with 7f211a9c360cca8c0526ed43298e8465426defbc7ceef103fd8728215bff5ac5 not found: ID does not exist" Jul 16 04:56:17.367282 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.367239 2561 scope.go:117] "RemoveContainer" containerID="33150fa1288ab8415dc6b6e27780a7479a2e7f9ad64f12e56355614bb3d5dca6" Jul 16 04:56:17.367434 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.367418 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33150fa1288ab8415dc6b6e27780a7479a2e7f9ad64f12e56355614bb3d5dca6"} err="failed to get container status \"33150fa1288ab8415dc6b6e27780a7479a2e7f9ad64f12e56355614bb3d5dca6\": rpc error: code = NotFound desc = could not find container \"33150fa1288ab8415dc6b6e27780a7479a2e7f9ad64f12e56355614bb3d5dca6\": container with ID starting with 33150fa1288ab8415dc6b6e27780a7479a2e7f9ad64f12e56355614bb3d5dca6 not found: ID does not exist" Jul 16 04:56:17.367478 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.367435 2561 scope.go:117] "RemoveContainer" containerID="03544c882793aa17807f36729eee45c6321182ebd9c84dcb5756a14d7e41a328" Jul 16 04:56:17.367621 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.367605 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03544c882793aa17807f36729eee45c6321182ebd9c84dcb5756a14d7e41a328"} err="failed to get container status \"03544c882793aa17807f36729eee45c6321182ebd9c84dcb5756a14d7e41a328\": rpc error: code = NotFound desc = could not find container \"03544c882793aa17807f36729eee45c6321182ebd9c84dcb5756a14d7e41a328\": container with ID starting with 03544c882793aa17807f36729eee45c6321182ebd9c84dcb5756a14d7e41a328 not found: ID does not exist" Jul 16 04:56:17.367706 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.367622 2561 scope.go:117] "RemoveContainer" containerID="15ada47c76554c150a3be6206a14e468731e4a8afa12b018706a632b4c204860" Jul 16 04:56:17.367885 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.367869 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15ada47c76554c150a3be6206a14e468731e4a8afa12b018706a632b4c204860"} err="failed to get container status \"15ada47c76554c150a3be6206a14e468731e4a8afa12b018706a632b4c204860\": rpc error: code = NotFound desc = could not find container \"15ada47c76554c150a3be6206a14e468731e4a8afa12b018706a632b4c204860\": container with ID starting with 15ada47c76554c150a3be6206a14e468731e4a8afa12b018706a632b4c204860 not found: ID does not exist" Jul 16 04:56:17.367928 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.367887 2561 scope.go:117] "RemoveContainer" containerID="a6ced93a8bf6f67c4966b02208ee257c825c3a9c22eb3d9af6bf4a1a881999c0" Jul 16 04:56:17.368088 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.368072 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6ced93a8bf6f67c4966b02208ee257c825c3a9c22eb3d9af6bf4a1a881999c0"} err="failed to get container status \"a6ced93a8bf6f67c4966b02208ee257c825c3a9c22eb3d9af6bf4a1a881999c0\": rpc error: code = NotFound desc = could not find container \"a6ced93a8bf6f67c4966b02208ee257c825c3a9c22eb3d9af6bf4a1a881999c0\": container with ID starting with a6ced93a8bf6f67c4966b02208ee257c825c3a9c22eb3d9af6bf4a1a881999c0 not found: ID does not exist" Jul 16 04:56:17.426541 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.426514 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1d4fafda-eab9-4056-9281-5dd0af7d8b74-alertmanager-main-db\") pod \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " Jul 16 04:56:17.426541 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.426544 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-kube-rbac-proxy-metric\") pod \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " Jul 16 04:56:17.426755 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.426579 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1d4fafda-eab9-4056-9281-5dd0af7d8b74-config-out\") pod \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " Jul 16 04:56:17.426755 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.426601 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-kube-rbac-proxy\") pod \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " Jul 16 04:56:17.426755 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.426622 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-web-config\") pod \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " Jul 16 04:56:17.426755 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.426650 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdw49\" (UniqueName: \"kubernetes.io/projected/1d4fafda-eab9-4056-9281-5dd0af7d8b74-kube-api-access-cdw49\") pod \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " Jul 16 04:56:17.426755 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.426675 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d4fafda-eab9-4056-9281-5dd0af7d8b74-alertmanager-trusted-ca-bundle\") pod \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " Jul 16 04:56:17.426755 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.426724 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-kube-rbac-proxy-web\") pod \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " Jul 16 04:56:17.426755 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.426749 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-cluster-tls-config\") pod \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " Jul 16 04:56:17.427157 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.426784 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1d4fafda-eab9-4056-9281-5dd0af7d8b74-tls-assets\") pod \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " Jul 16 04:56:17.427157 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.426810 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1d4fafda-eab9-4056-9281-5dd0af7d8b74-metrics-client-ca\") pod \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " Jul 16 04:56:17.427157 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.426839 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-config-volume\") pod \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " Jul 16 04:56:17.427157 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.426865 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-main-tls\") pod \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\" (UID: \"1d4fafda-eab9-4056-9281-5dd0af7d8b74\") " Jul 16 04:56:17.427157 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.426883 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d4fafda-eab9-4056-9281-5dd0af7d8b74-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "1d4fafda-eab9-4056-9281-5dd0af7d8b74" (UID: "1d4fafda-eab9-4056-9281-5dd0af7d8b74"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 04:56:17.427157 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.427140 2561 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1d4fafda-eab9-4056-9281-5dd0af7d8b74-alertmanager-main-db\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 04:56:17.427841 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.427808 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d4fafda-eab9-4056-9281-5dd0af7d8b74-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "1d4fafda-eab9-4056-9281-5dd0af7d8b74" (UID: "1d4fafda-eab9-4056-9281-5dd0af7d8b74"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:56:17.427945 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.427907 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d4fafda-eab9-4056-9281-5dd0af7d8b74-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "1d4fafda-eab9-4056-9281-5dd0af7d8b74" (UID: "1d4fafda-eab9-4056-9281-5dd0af7d8b74"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:56:17.429920 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.429616 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d4fafda-eab9-4056-9281-5dd0af7d8b74-config-out" (OuterVolumeSpecName: "config-out") pod "1d4fafda-eab9-4056-9281-5dd0af7d8b74" (UID: "1d4fafda-eab9-4056-9281-5dd0af7d8b74"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 04:56:17.430014 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.429977 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d4fafda-eab9-4056-9281-5dd0af7d8b74-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1d4fafda-eab9-4056-9281-5dd0af7d8b74" (UID: "1d4fafda-eab9-4056-9281-5dd0af7d8b74"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:56:17.430351 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.430278 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d4fafda-eab9-4056-9281-5dd0af7d8b74-kube-api-access-cdw49" (OuterVolumeSpecName: "kube-api-access-cdw49") pod "1d4fafda-eab9-4056-9281-5dd0af7d8b74" (UID: "1d4fafda-eab9-4056-9281-5dd0af7d8b74"). InnerVolumeSpecName "kube-api-access-cdw49". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:56:17.430473 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.430427 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "1d4fafda-eab9-4056-9281-5dd0af7d8b74" (UID: "1d4fafda-eab9-4056-9281-5dd0af7d8b74"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:56:17.430767 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.430739 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "1d4fafda-eab9-4056-9281-5dd0af7d8b74" (UID: "1d4fafda-eab9-4056-9281-5dd0af7d8b74"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:56:17.431011 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.430985 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "1d4fafda-eab9-4056-9281-5dd0af7d8b74" (UID: "1d4fafda-eab9-4056-9281-5dd0af7d8b74"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:56:17.431083 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.431003 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "1d4fafda-eab9-4056-9281-5dd0af7d8b74" (UID: "1d4fafda-eab9-4056-9281-5dd0af7d8b74"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:56:17.431123 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.431098 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-config-volume" (OuterVolumeSpecName: "config-volume") pod "1d4fafda-eab9-4056-9281-5dd0af7d8b74" (UID: "1d4fafda-eab9-4056-9281-5dd0af7d8b74"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:56:17.434827 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.434797 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "1d4fafda-eab9-4056-9281-5dd0af7d8b74" (UID: "1d4fafda-eab9-4056-9281-5dd0af7d8b74"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:56:17.441354 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.441332 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-web-config" (OuterVolumeSpecName: "web-config") pod "1d4fafda-eab9-4056-9281-5dd0af7d8b74" (UID: "1d4fafda-eab9-4056-9281-5dd0af7d8b74"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:56:17.528339 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.528304 2561 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1d4fafda-eab9-4056-9281-5dd0af7d8b74-tls-assets\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 04:56:17.528339 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.528333 2561 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1d4fafda-eab9-4056-9281-5dd0af7d8b74-metrics-client-ca\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 04:56:17.528524 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.528346 2561 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-config-volume\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 04:56:17.528524 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.528358 2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-main-tls\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 04:56:17.528524 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.528371 2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 04:56:17.528524 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.528383 2561 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1d4fafda-eab9-4056-9281-5dd0af7d8b74-config-out\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 04:56:17.528524 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.528394 2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 04:56:17.528524 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.528407 2561 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-web-config\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 04:56:17.528524 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.528417 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdw49\" (UniqueName: \"kubernetes.io/projected/1d4fafda-eab9-4056-9281-5dd0af7d8b74-kube-api-access-cdw49\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 04:56:17.528524 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.528429 2561 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d4fafda-eab9-4056-9281-5dd0af7d8b74-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 04:56:17.528524 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.528441 2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 04:56:17.528524 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.528453 2561 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1d4fafda-eab9-4056-9281-5dd0af7d8b74-cluster-tls-config\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 04:56:17.642554 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.642522 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 04:56:17.647297 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.647276 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 04:56:17.668852 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.668826 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 04:56:17.669099 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.669064 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="kube-rbac-proxy" Jul 16 04:56:17.669141 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.669102 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="kube-rbac-proxy" Jul 16 04:56:17.669141 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.669114 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="kube-rbac-proxy-metric" Jul 16 04:56:17.669141 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.669123 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="kube-rbac-proxy-metric" Jul 16 04:56:17.669141 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.669132 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="init-config-reloader" Jul 16 04:56:17.669141 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.669138 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="init-config-reloader" Jul 16 04:56:17.669288 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.669144 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="alertmanager" Jul 16 04:56:17.669288 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.669149 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="alertmanager" Jul 16 04:56:17.669288 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.669155 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="kube-rbac-proxy-web" Jul 16 04:56:17.669288 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.669159 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="kube-rbac-proxy-web" Jul 16 04:56:17.669288 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.669171 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="config-reloader" Jul 16 04:56:17.669288 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.669175 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="config-reloader" Jul 16 04:56:17.669288 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.669181 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="prom-label-proxy" Jul 16 04:56:17.669288 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.669189 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="prom-label-proxy" Jul 16 04:56:17.669288 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.669238 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="config-reloader" Jul 16 04:56:17.669288 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.669245 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="kube-rbac-proxy" Jul 16 04:56:17.669288 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.669249 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="prom-label-proxy" Jul 16 04:56:17.669288 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.669255 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="alertmanager" Jul 16 04:56:17.669288 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.669260 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="kube-rbac-proxy-web" Jul 16 04:56:17.669288 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.669266 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" containerName="kube-rbac-proxy-metric" Jul 16 04:56:17.671828 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.671810 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.674381 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.674360 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 16 04:56:17.674494 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.674377 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 16 04:56:17.674494 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.674359 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 16 04:56:17.674736 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.674719 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-s4f9d\"" Jul 16 04:56:17.674822 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.674741 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 16 04:56:17.674822 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.674742 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 16 04:56:17.674939 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.674871 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 16 04:56:17.674993 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.674960 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 16 04:56:17.675174 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.675153 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 16 04:56:17.679809 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.679789 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 16 04:56:17.685508 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.685487 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 04:56:17.830117 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.830076 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0f75d148-5b6a-4d27-949c-14e30653aaca-config-out\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.830280 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.830136 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0f75d148-5b6a-4d27-949c-14e30653aaca-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.830280 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.830157 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0f75d148-5b6a-4d27-949c-14e30653aaca-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.830280 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.830172 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0f75d148-5b6a-4d27-949c-14e30653aaca-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.830280 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.830188 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvtjf\" (UniqueName: \"kubernetes.io/projected/0f75d148-5b6a-4d27-949c-14e30653aaca-kube-api-access-tvtjf\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.830280 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.830208 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0f75d148-5b6a-4d27-949c-14e30653aaca-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.830280 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.830250 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0f75d148-5b6a-4d27-949c-14e30653aaca-web-config\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.830280 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.830275 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0f75d148-5b6a-4d27-949c-14e30653aaca-config-volume\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.830494 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.830299 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0f75d148-5b6a-4d27-949c-14e30653aaca-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.830494 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.830315 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0f75d148-5b6a-4d27-949c-14e30653aaca-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.830494 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.830339 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f75d148-5b6a-4d27-949c-14e30653aaca-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.830494 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.830362 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f75d148-5b6a-4d27-949c-14e30653aaca-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.830494 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.830390 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0f75d148-5b6a-4d27-949c-14e30653aaca-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.931758 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.931662 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0f75d148-5b6a-4d27-949c-14e30653aaca-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.931758 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.931734 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0f75d148-5b6a-4d27-949c-14e30653aaca-config-out\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.931934 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.931761 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0f75d148-5b6a-4d27-949c-14e30653aaca-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.931934 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.931781 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0f75d148-5b6a-4d27-949c-14e30653aaca-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.931934 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.931798 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0f75d148-5b6a-4d27-949c-14e30653aaca-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.931934 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.931817 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvtjf\" (UniqueName: \"kubernetes.io/projected/0f75d148-5b6a-4d27-949c-14e30653aaca-kube-api-access-tvtjf\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.931934 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.931840 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0f75d148-5b6a-4d27-949c-14e30653aaca-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.931934 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.931864 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0f75d148-5b6a-4d27-949c-14e30653aaca-web-config\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.931934 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.931901 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0f75d148-5b6a-4d27-949c-14e30653aaca-config-volume\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.931934 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.931931 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0f75d148-5b6a-4d27-949c-14e30653aaca-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.932273 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.931959 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0f75d148-5b6a-4d27-949c-14e30653aaca-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.932273 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.931996 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f75d148-5b6a-4d27-949c-14e30653aaca-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.932273 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.932020 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f75d148-5b6a-4d27-949c-14e30653aaca-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.932273 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.932132 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0f75d148-5b6a-4d27-949c-14e30653aaca-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.932915 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.932888 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f75d148-5b6a-4d27-949c-14e30653aaca-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.935155 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.935125 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0f75d148-5b6a-4d27-949c-14e30653aaca-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.935268 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.935123 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0f75d148-5b6a-4d27-949c-14e30653aaca-config-out\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.935268 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.935179 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0f75d148-5b6a-4d27-949c-14e30653aaca-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.935268 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.935235 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0f75d148-5b6a-4d27-949c-14e30653aaca-config-volume\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.935434 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.935273 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0f75d148-5b6a-4d27-949c-14e30653aaca-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.935434 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.935349 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0f75d148-5b6a-4d27-949c-14e30653aaca-web-config\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.935434 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.935386 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0f75d148-5b6a-4d27-949c-14e30653aaca-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.936047 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.936024 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f75d148-5b6a-4d27-949c-14e30653aaca-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.936211 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.936190 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0f75d148-5b6a-4d27-949c-14e30653aaca-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.937188 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.937172 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0f75d148-5b6a-4d27-949c-14e30653aaca-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.939292 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.939269 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvtjf\" (UniqueName: \"kubernetes.io/projected/0f75d148-5b6a-4d27-949c-14e30653aaca-kube-api-access-tvtjf\") pod \"alertmanager-main-0\" (UID: \"0f75d148-5b6a-4d27-949c-14e30653aaca\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:17.981405 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:17.981383 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:56:18.100192 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:18.100156 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d4fafda-eab9-4056-9281-5dd0af7d8b74" path="/var/lib/kubelet/pods/1d4fafda-eab9-4056-9281-5dd0af7d8b74/volumes" Jul 16 04:56:18.111136 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:18.111111 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 04:56:18.113245 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:56:18.113200 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f75d148_5b6a_4d27_949c_14e30653aaca.slice/crio-b8df0b5f0b2bc254ecfb01d888b4c56f6d1acc06e09d89a478318393c053300b WatchSource:0}: Error finding container b8df0b5f0b2bc254ecfb01d888b4c56f6d1acc06e09d89a478318393c053300b: Status 404 returned error can't find the container with id b8df0b5f0b2bc254ecfb01d888b4c56f6d1acc06e09d89a478318393c053300b Jul 16 04:56:18.322167 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:18.322137 2561 generic.go:356] "Generic (PLEG): container finished" podID="0f75d148-5b6a-4d27-949c-14e30653aaca" containerID="d233357c187a2b8ce8551e472d01d4a8e94ed6b1aa99e879714e42e211017a83" exitCode=0 Jul 16 04:56:18.322546 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:18.322174 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0f75d148-5b6a-4d27-949c-14e30653aaca","Type":"ContainerDied","Data":"d233357c187a2b8ce8551e472d01d4a8e94ed6b1aa99e879714e42e211017a83"} Jul 16 04:56:18.322546 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:18.322244 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0f75d148-5b6a-4d27-949c-14e30653aaca","Type":"ContainerStarted","Data":"b8df0b5f0b2bc254ecfb01d888b4c56f6d1acc06e09d89a478318393c053300b"} Jul 16 04:56:19.333525 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:19.333482 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0f75d148-5b6a-4d27-949c-14e30653aaca","Type":"ContainerStarted","Data":"3acaf16388b19b037a73efb2a82cea09b2009702103653d96ea1de095c8b775a"} Jul 16 04:56:19.333924 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:19.333573 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0f75d148-5b6a-4d27-949c-14e30653aaca","Type":"ContainerStarted","Data":"4e6121cde452bfcba30dc78a36b7229fef252e8f715f5eac0db635d2af62ba41"} Jul 16 04:56:19.333924 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:19.333598 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0f75d148-5b6a-4d27-949c-14e30653aaca","Type":"ContainerStarted","Data":"bde1b1da22ee797271e5a8d7cbf9dc3d3736d3ffa9dd2588f76fad998df3459e"} Jul 16 04:56:19.333924 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:19.333624 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0f75d148-5b6a-4d27-949c-14e30653aaca","Type":"ContainerStarted","Data":"6ed2445ca2664018b4feb96450873b538f30972d4f05419fb435929d8e9d63e5"} Jul 16 04:56:19.333924 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:19.333646 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0f75d148-5b6a-4d27-949c-14e30653aaca","Type":"ContainerStarted","Data":"640fbf20788a6265928532d75b655bf432ccdce011f70e16cca29eee0eaed600"} Jul 16 04:56:19.333924 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:19.333667 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0f75d148-5b6a-4d27-949c-14e30653aaca","Type":"ContainerStarted","Data":"cfb9db70229742074c10b77f5e6ecbcb403997bcd95cf8d3d226e1ff7f6ff96d"} Jul 16 04:56:19.361582 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:19.361483 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.343193357 podStartE2EDuration="2.361467005s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.273648ms" isStatefulPod=false podCreationTimestamp="2026-07-16 04:56:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:56:19.359558906 +0000 UTC m=+369.731682148" watchObservedRunningTime="2026-07-16 04:56:19.361467005 +0000 UTC m=+369.733590209" Jul 16 04:56:23.986264 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:23.986219 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs\") pod \"network-metrics-daemon-2srzm\" (UID: \"8b4ec3ce-70fc-4716-a3c6-5ab123a8be45\") " pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:56:23.988669 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:23.988643 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b4ec3ce-70fc-4716-a3c6-5ab123a8be45-metrics-certs\") pod \"network-metrics-daemon-2srzm\" (UID: \"8b4ec3ce-70fc-4716-a3c6-5ab123a8be45\") " pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:56:24.198487 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:24.198454 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vpdjw\"" Jul 16 04:56:24.206570 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:24.206550 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2srzm" Jul 16 04:56:24.329226 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:24.329203 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2srzm"] Jul 16 04:56:24.331932 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:56:24.331903 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b4ec3ce_70fc_4716_a3c6_5ab123a8be45.slice/crio-6ad0839fd64b6e9706e79a1fe26656dfd6a83da6e9ff03f1a1682e9bfe9056ed WatchSource:0}: Error finding container 6ad0839fd64b6e9706e79a1fe26656dfd6a83da6e9ff03f1a1682e9bfe9056ed: Status 404 returned error can't find the container with id 6ad0839fd64b6e9706e79a1fe26656dfd6a83da6e9ff03f1a1682e9bfe9056ed Jul 16 04:56:24.352423 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:24.352397 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2srzm" event={"ID":"8b4ec3ce-70fc-4716-a3c6-5ab123a8be45","Type":"ContainerStarted","Data":"6ad0839fd64b6e9706e79a1fe26656dfd6a83da6e9ff03f1a1682e9bfe9056ed"} Jul 16 04:56:25.357299 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:25.357259 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2srzm" event={"ID":"8b4ec3ce-70fc-4716-a3c6-5ab123a8be45","Type":"ContainerStarted","Data":"320ec2696414a571d4d563d28c23804f54e3f1ba60381a3a6bd11c0efab81d54"} Jul 16 04:56:25.357644 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:25.357323 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2srzm" event={"ID":"8b4ec3ce-70fc-4716-a3c6-5ab123a8be45","Type":"ContainerStarted","Data":"d56724ad6cdbc1a814b4af7087142cfe1cd31d5ea4d7fce75245262593ac40d0"} Jul 16 04:56:25.373372 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:25.373329 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2srzm" podStartSLOduration=374.578867379 podStartE2EDuration="6m15.373314224s" totalImagesPullingTime="794.446845ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:50:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:56:25.372603906 +0000 UTC m=+375.744727112" watchObservedRunningTime="2026-07-16 04:56:25.373314224 +0000 UTC m=+375.745437430" Jul 16 04:56:26.691401 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:26.691365 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54bcd668c4-mvt6j"] Jul 16 04:56:51.711538 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:51.711498 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-54bcd668c4-mvt6j" podUID="b5c61c5b-7313-42aa-a0ec-1340c883b107" containerName="console" containerID="cri-o://e56655060a962adf8c5e660084bc46039926f68dd04d1fedb9a3c3c617e579c6" gracePeriod=15 Jul 16 04:56:51.950091 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:51.950068 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:56:51.993938 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:51.993873 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5c61c5b-7313-42aa-a0ec-1340c883b107-trusted-ca-bundle\") pod \"b5c61c5b-7313-42aa-a0ec-1340c883b107\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " Jul 16 04:56:51.993938 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:51.993909 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b5c61c5b-7313-42aa-a0ec-1340c883b107-console-config\") pod \"b5c61c5b-7313-42aa-a0ec-1340c883b107\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " Jul 16 04:56:51.993938 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:51.993931 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b5c61c5b-7313-42aa-a0ec-1340c883b107-console-oauth-config\") pod \"b5c61c5b-7313-42aa-a0ec-1340c883b107\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " Jul 16 04:56:51.994145 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:51.993949 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4n2z\" (UniqueName: \"kubernetes.io/projected/b5c61c5b-7313-42aa-a0ec-1340c883b107-kube-api-access-x4n2z\") pod \"b5c61c5b-7313-42aa-a0ec-1340c883b107\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " Jul 16 04:56:51.994145 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:51.994039 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b5c61c5b-7313-42aa-a0ec-1340c883b107-oauth-serving-cert\") pod \"b5c61c5b-7313-42aa-a0ec-1340c883b107\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " Jul 16 04:56:51.994145 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:51.994067 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b5c61c5b-7313-42aa-a0ec-1340c883b107-service-ca\") pod \"b5c61c5b-7313-42aa-a0ec-1340c883b107\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " Jul 16 04:56:51.994145 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:51.994097 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b5c61c5b-7313-42aa-a0ec-1340c883b107-console-serving-cert\") pod \"b5c61c5b-7313-42aa-a0ec-1340c883b107\" (UID: \"b5c61c5b-7313-42aa-a0ec-1340c883b107\") " Jul 16 04:56:51.994436 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:51.994392 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5c61c5b-7313-42aa-a0ec-1340c883b107-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b5c61c5b-7313-42aa-a0ec-1340c883b107" (UID: "b5c61c5b-7313-42aa-a0ec-1340c883b107"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:56:51.994436 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:51.994426 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5c61c5b-7313-42aa-a0ec-1340c883b107-console-config" (OuterVolumeSpecName: "console-config") pod "b5c61c5b-7313-42aa-a0ec-1340c883b107" (UID: "b5c61c5b-7313-42aa-a0ec-1340c883b107"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:56:51.994605 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:51.994518 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5c61c5b-7313-42aa-a0ec-1340c883b107-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b5c61c5b-7313-42aa-a0ec-1340c883b107" (UID: "b5c61c5b-7313-42aa-a0ec-1340c883b107"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:56:51.994605 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:51.994537 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5c61c5b-7313-42aa-a0ec-1340c883b107-service-ca" (OuterVolumeSpecName: "service-ca") pod "b5c61c5b-7313-42aa-a0ec-1340c883b107" (UID: "b5c61c5b-7313-42aa-a0ec-1340c883b107"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:56:51.996426 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:51.996401 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5c61c5b-7313-42aa-a0ec-1340c883b107-kube-api-access-x4n2z" (OuterVolumeSpecName: "kube-api-access-x4n2z") pod "b5c61c5b-7313-42aa-a0ec-1340c883b107" (UID: "b5c61c5b-7313-42aa-a0ec-1340c883b107"). InnerVolumeSpecName "kube-api-access-x4n2z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:56:51.996508 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:51.996428 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5c61c5b-7313-42aa-a0ec-1340c883b107-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b5c61c5b-7313-42aa-a0ec-1340c883b107" (UID: "b5c61c5b-7313-42aa-a0ec-1340c883b107"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:56:51.996508 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:51.996472 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5c61c5b-7313-42aa-a0ec-1340c883b107-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b5c61c5b-7313-42aa-a0ec-1340c883b107" (UID: "b5c61c5b-7313-42aa-a0ec-1340c883b107"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:56:52.094583 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:52.094548 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b5c61c5b-7313-42aa-a0ec-1340c883b107-oauth-serving-cert\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 04:56:52.094583 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:52.094579 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b5c61c5b-7313-42aa-a0ec-1340c883b107-service-ca\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 04:56:52.094850 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:52.094591 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b5c61c5b-7313-42aa-a0ec-1340c883b107-console-serving-cert\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 04:56:52.094850 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:52.094604 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5c61c5b-7313-42aa-a0ec-1340c883b107-trusted-ca-bundle\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 04:56:52.094850 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:52.094618 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b5c61c5b-7313-42aa-a0ec-1340c883b107-console-config\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 04:56:52.094850 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:52.094657 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b5c61c5b-7313-42aa-a0ec-1340c883b107-console-oauth-config\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 04:56:52.094850 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:52.094670 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4n2z\" (UniqueName: \"kubernetes.io/projected/b5c61c5b-7313-42aa-a0ec-1340c883b107-kube-api-access-x4n2z\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 04:56:52.453532 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:52.453495 2561 generic.go:356] "Generic (PLEG): container finished" podID="b5c61c5b-7313-42aa-a0ec-1340c883b107" containerID="e56655060a962adf8c5e660084bc46039926f68dd04d1fedb9a3c3c617e579c6" exitCode=0 Jul 16 04:56:52.453532 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:52.453536 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54bcd668c4-mvt6j" event={"ID":"b5c61c5b-7313-42aa-a0ec-1340c883b107","Type":"ContainerDied","Data":"e56655060a962adf8c5e660084bc46039926f68dd04d1fedb9a3c3c617e579c6"} Jul 16 04:56:52.453775 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:52.453539 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54bcd668c4-mvt6j" Jul 16 04:56:52.453775 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:52.453576 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54bcd668c4-mvt6j" event={"ID":"b5c61c5b-7313-42aa-a0ec-1340c883b107","Type":"ContainerDied","Data":"0efa07ee98742b0dd349efbae6af38ea72a9301a2a1b2e193156876b80207a36"} Jul 16 04:56:52.453775 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:52.453597 2561 scope.go:117] "RemoveContainer" containerID="e56655060a962adf8c5e660084bc46039926f68dd04d1fedb9a3c3c617e579c6" Jul 16 04:56:52.461569 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:52.461528 2561 scope.go:117] "RemoveContainer" containerID="e56655060a962adf8c5e660084bc46039926f68dd04d1fedb9a3c3c617e579c6" Jul 16 04:56:52.461803 ip-10-0-135-187 kubenswrapper[2561]: E0716 04:56:52.461783 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e56655060a962adf8c5e660084bc46039926f68dd04d1fedb9a3c3c617e579c6\": container with ID starting with e56655060a962adf8c5e660084bc46039926f68dd04d1fedb9a3c3c617e579c6 not found: ID does not exist" containerID="e56655060a962adf8c5e660084bc46039926f68dd04d1fedb9a3c3c617e579c6" Jul 16 04:56:52.461852 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:52.461811 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e56655060a962adf8c5e660084bc46039926f68dd04d1fedb9a3c3c617e579c6"} err="failed to get container status \"e56655060a962adf8c5e660084bc46039926f68dd04d1fedb9a3c3c617e579c6\": rpc error: code = NotFound desc = could not find container \"e56655060a962adf8c5e660084bc46039926f68dd04d1fedb9a3c3c617e579c6\": container with ID starting with e56655060a962adf8c5e660084bc46039926f68dd04d1fedb9a3c3c617e579c6 not found: ID does not exist" Jul 16 04:56:52.470140 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:52.470118 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54bcd668c4-mvt6j"] Jul 16 04:56:52.472473 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:52.472452 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-54bcd668c4-mvt6j"] Jul 16 04:56:54.098897 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:56:54.098866 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5c61c5b-7313-42aa-a0ec-1340c883b107" path="/var/lib/kubelet/pods/b5c61c5b-7313-42aa-a0ec-1340c883b107/volumes" Jul 16 04:58:03.165148 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:03.165113 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cfv56"] Jul 16 04:58:03.165605 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:03.165358 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5c61c5b-7313-42aa-a0ec-1340c883b107" containerName="console" Jul 16 04:58:03.165605 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:03.165368 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5c61c5b-7313-42aa-a0ec-1340c883b107" containerName="console" Jul 16 04:58:03.165605 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:03.165418 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5c61c5b-7313-42aa-a0ec-1340c883b107" containerName="console" Jul 16 04:58:03.168172 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:03.168156 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cfv56" Jul 16 04:58:03.170889 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:03.170864 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 16 04:58:03.171017 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:03.170864 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-z4dzq\"" Jul 16 04:58:03.171017 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:03.170901 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 16 04:58:03.180377 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:03.180353 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cfv56"] Jul 16 04:58:03.295020 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:03.294980 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c35e58d-ef16-40d4-a97c-47f3ced31fba-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-cfv56\" (UID: \"0c35e58d-ef16-40d4-a97c-47f3ced31fba\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cfv56" Jul 16 04:58:03.295020 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:03.295023 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzwn9\" (UniqueName: \"kubernetes.io/projected/0c35e58d-ef16-40d4-a97c-47f3ced31fba-kube-api-access-wzwn9\") pod \"cert-manager-operator-controller-manager-55c444bf86-cfv56\" (UID: \"0c35e58d-ef16-40d4-a97c-47f3ced31fba\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cfv56" Jul 16 04:58:03.395679 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:03.395630 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c35e58d-ef16-40d4-a97c-47f3ced31fba-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-cfv56\" (UID: \"0c35e58d-ef16-40d4-a97c-47f3ced31fba\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cfv56" Jul 16 04:58:03.395856 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:03.395714 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzwn9\" (UniqueName: \"kubernetes.io/projected/0c35e58d-ef16-40d4-a97c-47f3ced31fba-kube-api-access-wzwn9\") pod \"cert-manager-operator-controller-manager-55c444bf86-cfv56\" (UID: \"0c35e58d-ef16-40d4-a97c-47f3ced31fba\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cfv56" Jul 16 04:58:03.396035 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:03.396019 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c35e58d-ef16-40d4-a97c-47f3ced31fba-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-cfv56\" (UID: \"0c35e58d-ef16-40d4-a97c-47f3ced31fba\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cfv56" Jul 16 04:58:03.406718 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:03.406676 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzwn9\" (UniqueName: \"kubernetes.io/projected/0c35e58d-ef16-40d4-a97c-47f3ced31fba-kube-api-access-wzwn9\") pod \"cert-manager-operator-controller-manager-55c444bf86-cfv56\" (UID: \"0c35e58d-ef16-40d4-a97c-47f3ced31fba\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cfv56" Jul 16 04:58:03.476807 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:03.476734 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cfv56" Jul 16 04:58:03.605617 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:03.605535 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cfv56"] Jul 16 04:58:03.608565 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:58:03.608536 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c35e58d_ef16_40d4_a97c_47f3ced31fba.slice/crio-52e63ffa70f8a231effc4811c8f21bcbb4ec6923be1082634d9f7087532f20c9 WatchSource:0}: Error finding container 52e63ffa70f8a231effc4811c8f21bcbb4ec6923be1082634d9f7087532f20c9: Status 404 returned error can't find the container with id 52e63ffa70f8a231effc4811c8f21bcbb4ec6923be1082634d9f7087532f20c9 Jul 16 04:58:03.693787 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:03.693751 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cfv56" event={"ID":"0c35e58d-ef16-40d4-a97c-47f3ced31fba","Type":"ContainerStarted","Data":"52e63ffa70f8a231effc4811c8f21bcbb4ec6923be1082634d9f7087532f20c9"} Jul 16 04:58:11.727172 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:11.727138 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cfv56" event={"ID":"0c35e58d-ef16-40d4-a97c-47f3ced31fba","Type":"ContainerStarted","Data":"0dc1692f5afe4286ac706757d9d82aae0b02f66cb4174fc6ff2ac0fef6fa92bd"} Jul 16 04:58:11.751308 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:11.751262 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cfv56" podStartSLOduration=1.368751683 podStartE2EDuration="8.751246515s" totalImagesPullingTime="7.382494832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:58:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:58:11.749128903 +0000 UTC m=+482.121252108" watchObservedRunningTime="2026-07-16 04:58:11.751246515 +0000 UTC m=+482.123369719" Jul 16 04:58:19.588132 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:19.588098 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-sw6bj"] Jul 16 04:58:19.591240 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:19.591223 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-sw6bj" Jul 16 04:58:19.593834 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:19.593807 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 16 04:58:19.593950 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:19.593859 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 16 04:58:19.593950 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:19.593914 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-xnpw5\"" Jul 16 04:58:19.598773 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:19.598748 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-sw6bj"] Jul 16 04:58:19.615980 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:19.615955 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl257\" (UniqueName: \"kubernetes.io/projected/52686d90-957e-4b50-86be-a026af264e37-kube-api-access-wl257\") pod \"cert-manager-webhook-6486ffd997-sw6bj\" (UID: \"52686d90-957e-4b50-86be-a026af264e37\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sw6bj" Jul 16 04:58:19.616075 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:19.615986 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/52686d90-957e-4b50-86be-a026af264e37-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-sw6bj\" (UID: \"52686d90-957e-4b50-86be-a026af264e37\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sw6bj" Jul 16 04:58:19.717142 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:19.717110 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl257\" (UniqueName: \"kubernetes.io/projected/52686d90-957e-4b50-86be-a026af264e37-kube-api-access-wl257\") pod \"cert-manager-webhook-6486ffd997-sw6bj\" (UID: \"52686d90-957e-4b50-86be-a026af264e37\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sw6bj" Jul 16 04:58:19.717142 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:19.717143 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/52686d90-957e-4b50-86be-a026af264e37-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-sw6bj\" (UID: \"52686d90-957e-4b50-86be-a026af264e37\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sw6bj" Jul 16 04:58:19.724764 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:19.724741 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/52686d90-957e-4b50-86be-a026af264e37-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-sw6bj\" (UID: \"52686d90-957e-4b50-86be-a026af264e37\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sw6bj" Jul 16 04:58:19.724952 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:19.724931 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl257\" (UniqueName: \"kubernetes.io/projected/52686d90-957e-4b50-86be-a026af264e37-kube-api-access-wl257\") pod \"cert-manager-webhook-6486ffd997-sw6bj\" (UID: \"52686d90-957e-4b50-86be-a026af264e37\") " pod="cert-manager/cert-manager-webhook-6486ffd997-sw6bj" Jul 16 04:58:19.900787 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:19.900678 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-sw6bj" Jul 16 04:58:20.025600 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:20.025569 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-sw6bj"] Jul 16 04:58:20.028203 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:58:20.028179 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52686d90_957e_4b50_86be_a026af264e37.slice/crio-86deb1c3408f931fc014949cbc5fb3895840a57366d85f63cb2b0ea0f4b9c5b2 WatchSource:0}: Error finding container 86deb1c3408f931fc014949cbc5fb3895840a57366d85f63cb2b0ea0f4b9c5b2: Status 404 returned error can't find the container with id 86deb1c3408f931fc014949cbc5fb3895840a57366d85f63cb2b0ea0f4b9c5b2 Jul 16 04:58:20.769331 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:20.769299 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-sw6bj" event={"ID":"52686d90-957e-4b50-86be-a026af264e37","Type":"ContainerStarted","Data":"86deb1c3408f931fc014949cbc5fb3895840a57366d85f63cb2b0ea0f4b9c5b2"} Jul 16 04:58:23.781801 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:23.781765 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-sw6bj" event={"ID":"52686d90-957e-4b50-86be-a026af264e37","Type":"ContainerStarted","Data":"e0853f50075ea5b17ac38a7148822b04a9b7b39d69f22e24df923c093a9eb542"} Jul 16 04:58:23.782171 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:23.781821 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-sw6bj" Jul 16 04:58:23.797297 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:23.797250 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-sw6bj" podStartSLOduration=2.017414036 podStartE2EDuration="4.797236666s" totalImagesPullingTime="2.77982263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:58:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:58:23.796221712 +0000 UTC m=+494.168344908" watchObservedRunningTime="2026-07-16 04:58:23.797236666 +0000 UTC m=+494.169359871" Jul 16 04:58:29.787639 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:29.787610 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-sw6bj" Jul 16 04:58:49.109152 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.109113 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z"] Jul 16 04:58:49.113476 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.113453 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z" Jul 16 04:58:49.116293 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.116256 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 16 04:58:49.116293 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.116287 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 16 04:58:49.116612 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.116595 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 16 04:58:49.116663 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.116627 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 16 04:58:49.116978 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.116957 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4rlvf\"" Jul 16 04:58:49.127482 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.127458 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z"] Jul 16 04:58:49.244739 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.244697 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/408adabb-1c69-46f7-8c1b-de7b15045259-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dddb677c8-5d67z\" (UID: \"408adabb-1c69-46f7-8c1b-de7b15045259\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z" Jul 16 04:58:49.244739 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.244739 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/408adabb-1c69-46f7-8c1b-de7b15045259-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dddb677c8-5d67z\" (UID: \"408adabb-1c69-46f7-8c1b-de7b15045259\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z" Jul 16 04:58:49.244949 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.244813 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/408adabb-1c69-46f7-8c1b-de7b15045259-manifests\") pod \"opendatahub-operator-controller-manager-6dddb677c8-5d67z\" (UID: \"408adabb-1c69-46f7-8c1b-de7b15045259\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z" Jul 16 04:58:49.244949 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.244866 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d447r\" (UniqueName: \"kubernetes.io/projected/408adabb-1c69-46f7-8c1b-de7b15045259-kube-api-access-d447r\") pod \"opendatahub-operator-controller-manager-6dddb677c8-5d67z\" (UID: \"408adabb-1c69-46f7-8c1b-de7b15045259\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z" Jul 16 04:58:49.244949 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.244896 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/408adabb-1c69-46f7-8c1b-de7b15045259-tmp\") pod \"opendatahub-operator-controller-manager-6dddb677c8-5d67z\" (UID: \"408adabb-1c69-46f7-8c1b-de7b15045259\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z" Jul 16 04:58:49.345745 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.345710 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d447r\" (UniqueName: \"kubernetes.io/projected/408adabb-1c69-46f7-8c1b-de7b15045259-kube-api-access-d447r\") pod \"opendatahub-operator-controller-manager-6dddb677c8-5d67z\" (UID: \"408adabb-1c69-46f7-8c1b-de7b15045259\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z" Jul 16 04:58:49.345916 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.345757 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/408adabb-1c69-46f7-8c1b-de7b15045259-tmp\") pod \"opendatahub-operator-controller-manager-6dddb677c8-5d67z\" (UID: \"408adabb-1c69-46f7-8c1b-de7b15045259\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z" Jul 16 04:58:49.345916 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.345787 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/408adabb-1c69-46f7-8c1b-de7b15045259-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dddb677c8-5d67z\" (UID: \"408adabb-1c69-46f7-8c1b-de7b15045259\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z" Jul 16 04:58:49.345916 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.345808 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/408adabb-1c69-46f7-8c1b-de7b15045259-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dddb677c8-5d67z\" (UID: \"408adabb-1c69-46f7-8c1b-de7b15045259\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z" Jul 16 04:58:49.345916 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.345836 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/408adabb-1c69-46f7-8c1b-de7b15045259-manifests\") pod \"opendatahub-operator-controller-manager-6dddb677c8-5d67z\" (UID: \"408adabb-1c69-46f7-8c1b-de7b15045259\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z" Jul 16 04:58:49.346170 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.346153 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/408adabb-1c69-46f7-8c1b-de7b15045259-manifests\") pod \"opendatahub-operator-controller-manager-6dddb677c8-5d67z\" (UID: \"408adabb-1c69-46f7-8c1b-de7b15045259\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z" Jul 16 04:58:49.348239 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.348216 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/408adabb-1c69-46f7-8c1b-de7b15045259-tmp\") pod \"opendatahub-operator-controller-manager-6dddb677c8-5d67z\" (UID: \"408adabb-1c69-46f7-8c1b-de7b15045259\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z" Jul 16 04:58:49.348522 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.348500 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/408adabb-1c69-46f7-8c1b-de7b15045259-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dddb677c8-5d67z\" (UID: \"408adabb-1c69-46f7-8c1b-de7b15045259\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z" Jul 16 04:58:49.348625 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.348595 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/408adabb-1c69-46f7-8c1b-de7b15045259-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dddb677c8-5d67z\" (UID: \"408adabb-1c69-46f7-8c1b-de7b15045259\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z" Jul 16 04:58:49.353734 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.353716 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d447r\" (UniqueName: \"kubernetes.io/projected/408adabb-1c69-46f7-8c1b-de7b15045259-kube-api-access-d447r\") pod \"opendatahub-operator-controller-manager-6dddb677c8-5d67z\" (UID: \"408adabb-1c69-46f7-8c1b-de7b15045259\") " pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z" Jul 16 04:58:49.427910 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.427824 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z" Jul 16 04:58:49.563349 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:58:49.563316 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod408adabb_1c69_46f7_8c1b_de7b15045259.slice/crio-5e50b7dbea97dd0abda3b0e822c7b910ab2a23942e1fdcd56b6cb957a6b36b81 WatchSource:0}: Error finding container 5e50b7dbea97dd0abda3b0e822c7b910ab2a23942e1fdcd56b6cb957a6b36b81: Status 404 returned error can't find the container with id 5e50b7dbea97dd0abda3b0e822c7b910ab2a23942e1fdcd56b6cb957a6b36b81 Jul 16 04:58:49.563506 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.563484 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z"] Jul 16 04:58:49.878743 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:49.878708 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z" event={"ID":"408adabb-1c69-46f7-8c1b-de7b15045259","Type":"ContainerStarted","Data":"5e50b7dbea97dd0abda3b0e822c7b910ab2a23942e1fdcd56b6cb957a6b36b81"} Jul 16 04:58:52.892978 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:52.892943 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z" event={"ID":"408adabb-1c69-46f7-8c1b-de7b15045259","Type":"ContainerStarted","Data":"2c38e210054544f084e01fb201fdc905d20e6f1e6b3a77d2b297daf1d4666686"} Jul 16 04:58:53.898415 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:53.898322 2561 generic.go:356] "Generic (PLEG): container finished" podID="408adabb-1c69-46f7-8c1b-de7b15045259" containerID="2c38e210054544f084e01fb201fdc905d20e6f1e6b3a77d2b297daf1d4666686" exitCode=0 Jul 16 04:58:53.898415 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:53.898382 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z" event={"ID":"408adabb-1c69-46f7-8c1b-de7b15045259","Type":"ContainerDied","Data":"2c38e210054544f084e01fb201fdc905d20e6f1e6b3a77d2b297daf1d4666686"} Jul 16 04:58:54.905462 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:54.905420 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z" event={"ID":"408adabb-1c69-46f7-8c1b-de7b15045259","Type":"ContainerStarted","Data":"3ce6b88f1cdd278527d3876a216cf834bc209230c8b2baa631e5427a42c3ecd3"} Jul 16 04:58:54.905945 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:54.905546 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z" Jul 16 04:58:54.925345 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:58:54.925303 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z" podStartSLOduration=1.645255148 podStartE2EDuration="5.925289677s" totalImagesPullingTime="3.222122747s" totalInitContainerRuntime="1.057911782s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:58:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:58:54.923588268 +0000 UTC m=+525.295711510" watchObservedRunningTime="2026-07-16 04:58:54.925289677 +0000 UTC m=+525.297412902" Jul 16 04:59:05.911894 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:05.911818 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6dddb677c8-5d67z" Jul 16 04:59:06.447741 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:06.447705 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-8647fcdd7d-6kck6"] Jul 16 04:59:06.450821 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:06.450803 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8647fcdd7d-6kck6" Jul 16 04:59:06.453588 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:06.453568 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 16 04:59:06.453700 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:06.453566 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 16 04:59:06.454668 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:06.454641 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-chx7n\"" Jul 16 04:59:06.454793 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:06.454641 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 16 04:59:06.454793 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:06.454710 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 16 04:59:06.454793 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:06.454727 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 16 04:59:06.460544 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:06.460524 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8647fcdd7d-6kck6"] Jul 16 04:59:06.582672 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:06.582636 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae569a1c-470c-41de-984c-ee5536109cae-cert\") pod \"lws-controller-manager-8647fcdd7d-6kck6\" (UID: \"ae569a1c-470c-41de-984c-ee5536109cae\") " pod="openshift-lws-operator/lws-controller-manager-8647fcdd7d-6kck6" Jul 16 04:59:06.582877 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:06.582675 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ae569a1c-470c-41de-984c-ee5536109cae-manager-config\") pod \"lws-controller-manager-8647fcdd7d-6kck6\" (UID: \"ae569a1c-470c-41de-984c-ee5536109cae\") " pod="openshift-lws-operator/lws-controller-manager-8647fcdd7d-6kck6" Jul 16 04:59:06.582877 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:06.582757 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7kv2\" (UniqueName: \"kubernetes.io/projected/ae569a1c-470c-41de-984c-ee5536109cae-kube-api-access-q7kv2\") pod \"lws-controller-manager-8647fcdd7d-6kck6\" (UID: \"ae569a1c-470c-41de-984c-ee5536109cae\") " pod="openshift-lws-operator/lws-controller-manager-8647fcdd7d-6kck6" Jul 16 04:59:06.582877 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:06.582802 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ae569a1c-470c-41de-984c-ee5536109cae-metrics-cert\") pod \"lws-controller-manager-8647fcdd7d-6kck6\" (UID: \"ae569a1c-470c-41de-984c-ee5536109cae\") " pod="openshift-lws-operator/lws-controller-manager-8647fcdd7d-6kck6" Jul 16 04:59:06.683446 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:06.683412 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae569a1c-470c-41de-984c-ee5536109cae-cert\") pod \"lws-controller-manager-8647fcdd7d-6kck6\" (UID: \"ae569a1c-470c-41de-984c-ee5536109cae\") " pod="openshift-lws-operator/lws-controller-manager-8647fcdd7d-6kck6" Jul 16 04:59:06.683595 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:06.683454 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ae569a1c-470c-41de-984c-ee5536109cae-manager-config\") pod \"lws-controller-manager-8647fcdd7d-6kck6\" (UID: \"ae569a1c-470c-41de-984c-ee5536109cae\") " pod="openshift-lws-operator/lws-controller-manager-8647fcdd7d-6kck6" Jul 16 04:59:06.683595 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:06.683483 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7kv2\" (UniqueName: \"kubernetes.io/projected/ae569a1c-470c-41de-984c-ee5536109cae-kube-api-access-q7kv2\") pod \"lws-controller-manager-8647fcdd7d-6kck6\" (UID: \"ae569a1c-470c-41de-984c-ee5536109cae\") " pod="openshift-lws-operator/lws-controller-manager-8647fcdd7d-6kck6" Jul 16 04:59:06.683595 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:06.683513 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ae569a1c-470c-41de-984c-ee5536109cae-metrics-cert\") pod \"lws-controller-manager-8647fcdd7d-6kck6\" (UID: \"ae569a1c-470c-41de-984c-ee5536109cae\") " pod="openshift-lws-operator/lws-controller-manager-8647fcdd7d-6kck6" Jul 16 04:59:06.684097 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:06.684068 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ae569a1c-470c-41de-984c-ee5536109cae-manager-config\") pod \"lws-controller-manager-8647fcdd7d-6kck6\" (UID: \"ae569a1c-470c-41de-984c-ee5536109cae\") " pod="openshift-lws-operator/lws-controller-manager-8647fcdd7d-6kck6" Jul 16 04:59:06.686145 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:06.686122 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ae569a1c-470c-41de-984c-ee5536109cae-metrics-cert\") pod \"lws-controller-manager-8647fcdd7d-6kck6\" (UID: \"ae569a1c-470c-41de-984c-ee5536109cae\") " pod="openshift-lws-operator/lws-controller-manager-8647fcdd7d-6kck6" Jul 16 04:59:06.686232 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:06.686162 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae569a1c-470c-41de-984c-ee5536109cae-cert\") pod \"lws-controller-manager-8647fcdd7d-6kck6\" (UID: \"ae569a1c-470c-41de-984c-ee5536109cae\") " pod="openshift-lws-operator/lws-controller-manager-8647fcdd7d-6kck6" Jul 16 04:59:06.691749 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:06.691723 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7kv2\" (UniqueName: \"kubernetes.io/projected/ae569a1c-470c-41de-984c-ee5536109cae-kube-api-access-q7kv2\") pod \"lws-controller-manager-8647fcdd7d-6kck6\" (UID: \"ae569a1c-470c-41de-984c-ee5536109cae\") " pod="openshift-lws-operator/lws-controller-manager-8647fcdd7d-6kck6" Jul 16 04:59:06.759598 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:06.759521 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8647fcdd7d-6kck6" Jul 16 04:59:06.902837 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:06.902812 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8647fcdd7d-6kck6"] Jul 16 04:59:06.905159 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:59:06.905134 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae569a1c_470c_41de_984c_ee5536109cae.slice/crio-66e59d4c7cde1ec2e13d146fe704add17020aa228c89e80942a425960994e761 WatchSource:0}: Error finding container 66e59d4c7cde1ec2e13d146fe704add17020aa228c89e80942a425960994e761: Status 404 returned error can't find the container with id 66e59d4c7cde1ec2e13d146fe704add17020aa228c89e80942a425960994e761 Jul 16 04:59:06.954280 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:06.954246 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8647fcdd7d-6kck6" event={"ID":"ae569a1c-470c-41de-984c-ee5536109cae","Type":"ContainerStarted","Data":"66e59d4c7cde1ec2e13d146fe704add17020aa228c89e80942a425960994e761"} Jul 16 04:59:09.970197 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:09.970156 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8647fcdd7d-6kck6" event={"ID":"ae569a1c-470c-41de-984c-ee5536109cae","Type":"ContainerStarted","Data":"5fd2fa156f3ae0e42988e3206f54b8b429273a28e48cd8b2d2080792a54fe161"} Jul 16 04:59:09.970723 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:09.970242 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-8647fcdd7d-6kck6" Jul 16 04:59:09.988229 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:09.988170 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-8647fcdd7d-6kck6" podStartSLOduration=1.386610081 podStartE2EDuration="3.988153635s" totalImagesPullingTime="2.601543554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:59:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:59:09.986248479 +0000 UTC m=+540.358371685" watchObservedRunningTime="2026-07-16 04:59:09.988153635 +0000 UTC m=+540.360276839" Jul 16 04:59:15.453905 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.453868 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k"] Jul 16 04:59:15.458536 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.458513 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.461371 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.461348 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 16 04:59:15.461535 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.461386 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 16 04:59:15.461648 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.461632 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 04:59:15.461771 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.461747 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-gnst8\"" Jul 16 04:59:15.467746 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.467725 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k"] Jul 16 04:59:15.552335 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.552292 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d36cdcda-f986-4d57-a141-12b5c455e8db-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.552335 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.552341 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmbk7\" (UniqueName: \"kubernetes.io/projected/d36cdcda-f986-4d57-a141-12b5c455e8db-kube-api-access-qmbk7\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.552540 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.552389 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36cdcda-f986-4d57-a141-12b5c455e8db-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.552540 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.552406 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d36cdcda-f986-4d57-a141-12b5c455e8db-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.552540 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.552437 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d36cdcda-f986-4d57-a141-12b5c455e8db-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.552540 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.552472 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d36cdcda-f986-4d57-a141-12b5c455e8db-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.552540 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.552490 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d36cdcda-f986-4d57-a141-12b5c455e8db-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.552540 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.552522 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d36cdcda-f986-4d57-a141-12b5c455e8db-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.552757 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.552546 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36cdcda-f986-4d57-a141-12b5c455e8db-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.653745 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.653711 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d36cdcda-f986-4d57-a141-12b5c455e8db-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.653901 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.653748 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d36cdcda-f986-4d57-a141-12b5c455e8db-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.653901 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.653777 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d36cdcda-f986-4d57-a141-12b5c455e8db-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.653990 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.653900 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36cdcda-f986-4d57-a141-12b5c455e8db-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.653990 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.653959 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d36cdcda-f986-4d57-a141-12b5c455e8db-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.654070 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.653991 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmbk7\" (UniqueName: \"kubernetes.io/projected/d36cdcda-f986-4d57-a141-12b5c455e8db-kube-api-access-qmbk7\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.654070 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.654045 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36cdcda-f986-4d57-a141-12b5c455e8db-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.654151 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.654070 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d36cdcda-f986-4d57-a141-12b5c455e8db-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.654151 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.654100 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d36cdcda-f986-4d57-a141-12b5c455e8db-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.654151 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.654144 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d36cdcda-f986-4d57-a141-12b5c455e8db-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.654283 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.654197 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d36cdcda-f986-4d57-a141-12b5c455e8db-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.654346 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.654326 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36cdcda-f986-4d57-a141-12b5c455e8db-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.654459 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.654437 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36cdcda-f986-4d57-a141-12b5c455e8db-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.654706 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.654670 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d36cdcda-f986-4d57-a141-12b5c455e8db-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.656307 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.656288 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d36cdcda-f986-4d57-a141-12b5c455e8db-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.656730 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.656708 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d36cdcda-f986-4d57-a141-12b5c455e8db-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.663608 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.663584 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmbk7\" (UniqueName: \"kubernetes.io/projected/d36cdcda-f986-4d57-a141-12b5c455e8db-kube-api-access-qmbk7\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.664096 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.664079 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d36cdcda-f986-4d57-a141-12b5c455e8db-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678z892k\" (UID: \"d36cdcda-f986-4d57-a141-12b5c455e8db\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.770138 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.770054 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:15.901592 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:15.901481 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k"] Jul 16 04:59:15.904301 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:59:15.904273 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd36cdcda_f986_4d57_a141_12b5c455e8db.slice/crio-37f8f72944199e3bb8c0a1fd68245d206f0662a4970a91f6a246a675ac359010 WatchSource:0}: Error finding container 37f8f72944199e3bb8c0a1fd68245d206f0662a4970a91f6a246a675ac359010: Status 404 returned error can't find the container with id 37f8f72944199e3bb8c0a1fd68245d206f0662a4970a91f6a246a675ac359010 Jul 16 04:59:16.001544 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.001505 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" event={"ID":"d36cdcda-f986-4d57-a141-12b5c455e8db","Type":"ContainerStarted","Data":"37f8f72944199e3bb8c0a1fd68245d206f0662a4970a91f6a246a675ac359010"} Jul 16 04:59:16.100666 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.100636 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n"] Jul 16 04:59:16.104902 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.104883 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" Jul 16 04:59:16.107607 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.107584 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 16 04:59:16.107607 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.107612 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 16 04:59:16.107842 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.107654 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-wd4vg\"" Jul 16 04:59:16.107903 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.107891 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 16 04:59:16.116420 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.116399 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n"] Jul 16 04:59:16.158630 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.158602 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3ab1b99e-8fcc-400e-9537-357fa0e6a5c1-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-prk6n\" (UID: \"3ab1b99e-8fcc-400e-9537-357fa0e6a5c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" Jul 16 04:59:16.158630 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.158640 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3ab1b99e-8fcc-400e-9537-357fa0e6a5c1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-prk6n\" (UID: \"3ab1b99e-8fcc-400e-9537-357fa0e6a5c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" Jul 16 04:59:16.158850 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.158667 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ab1b99e-8fcc-400e-9537-357fa0e6a5c1-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-prk6n\" (UID: \"3ab1b99e-8fcc-400e-9537-357fa0e6a5c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" Jul 16 04:59:16.158850 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.158703 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfgbj\" (UniqueName: \"kubernetes.io/projected/3ab1b99e-8fcc-400e-9537-357fa0e6a5c1-kube-api-access-wfgbj\") pod \"istiod-openshift-gateway-94bb8fbfd-prk6n\" (UID: \"3ab1b99e-8fcc-400e-9537-357fa0e6a5c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" Jul 16 04:59:16.158850 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.158774 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3ab1b99e-8fcc-400e-9537-357fa0e6a5c1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-prk6n\" (UID: \"3ab1b99e-8fcc-400e-9537-357fa0e6a5c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" Jul 16 04:59:16.158850 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.158816 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ab1b99e-8fcc-400e-9537-357fa0e6a5c1-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-prk6n\" (UID: \"3ab1b99e-8fcc-400e-9537-357fa0e6a5c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" Jul 16 04:59:16.158976 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.158852 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ab1b99e-8fcc-400e-9537-357fa0e6a5c1-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-prk6n\" (UID: \"3ab1b99e-8fcc-400e-9537-357fa0e6a5c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" Jul 16 04:59:16.260074 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.260044 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3ab1b99e-8fcc-400e-9537-357fa0e6a5c1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-prk6n\" (UID: \"3ab1b99e-8fcc-400e-9537-357fa0e6a5c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" Jul 16 04:59:16.260249 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.260082 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ab1b99e-8fcc-400e-9537-357fa0e6a5c1-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-prk6n\" (UID: \"3ab1b99e-8fcc-400e-9537-357fa0e6a5c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" Jul 16 04:59:16.260249 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.260115 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ab1b99e-8fcc-400e-9537-357fa0e6a5c1-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-prk6n\" (UID: \"3ab1b99e-8fcc-400e-9537-357fa0e6a5c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" Jul 16 04:59:16.260249 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.260147 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3ab1b99e-8fcc-400e-9537-357fa0e6a5c1-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-prk6n\" (UID: \"3ab1b99e-8fcc-400e-9537-357fa0e6a5c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" Jul 16 04:59:16.260249 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.260174 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3ab1b99e-8fcc-400e-9537-357fa0e6a5c1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-prk6n\" (UID: \"3ab1b99e-8fcc-400e-9537-357fa0e6a5c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" Jul 16 04:59:16.260249 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.260204 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ab1b99e-8fcc-400e-9537-357fa0e6a5c1-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-prk6n\" (UID: \"3ab1b99e-8fcc-400e-9537-357fa0e6a5c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" Jul 16 04:59:16.260249 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.260227 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfgbj\" (UniqueName: \"kubernetes.io/projected/3ab1b99e-8fcc-400e-9537-357fa0e6a5c1-kube-api-access-wfgbj\") pod \"istiod-openshift-gateway-94bb8fbfd-prk6n\" (UID: \"3ab1b99e-8fcc-400e-9537-357fa0e6a5c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" Jul 16 04:59:16.261000 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.260937 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3ab1b99e-8fcc-400e-9537-357fa0e6a5c1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-prk6n\" (UID: \"3ab1b99e-8fcc-400e-9537-357fa0e6a5c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" Jul 16 04:59:16.262652 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.262635 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ab1b99e-8fcc-400e-9537-357fa0e6a5c1-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-prk6n\" (UID: \"3ab1b99e-8fcc-400e-9537-357fa0e6a5c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" Jul 16 04:59:16.262759 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.262704 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3ab1b99e-8fcc-400e-9537-357fa0e6a5c1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-prk6n\" (UID: \"3ab1b99e-8fcc-400e-9537-357fa0e6a5c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" Jul 16 04:59:16.262934 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.262913 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ab1b99e-8fcc-400e-9537-357fa0e6a5c1-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-prk6n\" (UID: \"3ab1b99e-8fcc-400e-9537-357fa0e6a5c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" Jul 16 04:59:16.263009 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.262941 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3ab1b99e-8fcc-400e-9537-357fa0e6a5c1-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-prk6n\" (UID: \"3ab1b99e-8fcc-400e-9537-357fa0e6a5c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" Jul 16 04:59:16.271424 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.271401 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ab1b99e-8fcc-400e-9537-357fa0e6a5c1-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-prk6n\" (UID: \"3ab1b99e-8fcc-400e-9537-357fa0e6a5c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" Jul 16 04:59:16.271504 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.271453 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfgbj\" (UniqueName: \"kubernetes.io/projected/3ab1b99e-8fcc-400e-9537-357fa0e6a5c1-kube-api-access-wfgbj\") pod \"istiod-openshift-gateway-94bb8fbfd-prk6n\" (UID: \"3ab1b99e-8fcc-400e-9537-357fa0e6a5c1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" Jul 16 04:59:16.414717 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.414609 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" Jul 16 04:59:16.550511 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:16.550484 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n"] Jul 16 04:59:16.552390 ip-10-0-135-187 kubenswrapper[2561]: W0716 04:59:16.552363 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ab1b99e_8fcc_400e_9537_357fa0e6a5c1.slice/crio-f34d538472decb1e85bd5bb846f63c73ceb6bccf62ad6446d8b271e2faedf0df WatchSource:0}: Error finding container f34d538472decb1e85bd5bb846f63c73ceb6bccf62ad6446d8b271e2faedf0df: Status 404 returned error can't find the container with id f34d538472decb1e85bd5bb846f63c73ceb6bccf62ad6446d8b271e2faedf0df Jul 16 04:59:17.006624 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:17.006587 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" event={"ID":"3ab1b99e-8fcc-400e-9537-357fa0e6a5c1","Type":"ContainerStarted","Data":"f34d538472decb1e85bd5bb846f63c73ceb6bccf62ad6446d8b271e2faedf0df"} Jul 16 04:59:18.880298 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:18.880243 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 04:59:18.880596 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:18.880343 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 04:59:19.019004 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:19.018967 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" event={"ID":"3ab1b99e-8fcc-400e-9537-357fa0e6a5c1","Type":"ContainerStarted","Data":"3f2ddb00c2bd1179e98ba5b19b9e16c9e44dbe2a27edc80156a3f5a544d1fb70"} Jul 16 04:59:19.019172 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:19.019139 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" Jul 16 04:59:19.020976 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:19.020942 2561 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-prk6n container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 04:59:19.021107 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:19.021056 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" podUID="3ab1b99e-8fcc-400e-9537-357fa0e6a5c1" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 04:59:19.040909 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:19.040855 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" podStartSLOduration=0.715330817 podStartE2EDuration="3.040838402s" totalImagesPullingTime="2.325507585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:59:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:59:19.040605869 +0000 UTC m=+549.412729101" watchObservedRunningTime="2026-07-16 04:59:19.040838402 +0000 UTC m=+549.412961589" Jul 16 04:59:20.025129 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:20.025096 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-prk6n" Jul 16 04:59:20.865355 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:20.865324 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 04:59:20.865413 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:20.865398 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 04:59:20.865443 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:20.865428 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 04:59:20.976849 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:20.976819 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-8647fcdd7d-6kck6" Jul 16 04:59:21.030794 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:21.030757 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" event={"ID":"d36cdcda-f986-4d57-a141-12b5c455e8db","Type":"ContainerStarted","Data":"bd641a411862fdffd6f9f98571e1d5c76c3bbccf2c6ceeb955bffaaf00d6e36e"} Jul 16 04:59:21.053732 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:21.053661 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" podStartSLOduration=1.094702133 podStartE2EDuration="6.053643256s" totalImagesPullingTime="4.958941123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:59:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:59:21.052170587 +0000 UTC m=+551.424293806" watchObservedRunningTime="2026-07-16 04:59:21.053643256 +0000 UTC m=+551.425766461" Jul 16 04:59:21.770513 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:21.770479 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:21.775137 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:21.775113 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:22.034857 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:22.034780 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 04:59:22.035837 ip-10-0-135-187 kubenswrapper[2561]: I0716 04:59:22.035819 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678z892k" Jul 16 05:01:53.924307 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:53.924279 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x"] Jul 16 05:01:53.927422 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:53.927405 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:53.930035 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:53.930014 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-8bnp5\"" Jul 16 05:01:53.936522 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:53.936500 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x"] Jul 16 05:01:53.975938 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:53.975905 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fda6774-a044-441f-8357-afec2ddf322e-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:53.976077 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:53.975941 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l524l\" (UniqueName: \"kubernetes.io/projected/4fda6774-a044-441f-8357-afec2ddf322e-kube-api-access-l524l\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:53.976077 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:53.975965 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fda6774-a044-441f-8357-afec2ddf322e-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:53.976077 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:53.976013 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fda6774-a044-441f-8357-afec2ddf322e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:53.976077 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:53.976034 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fda6774-a044-441f-8357-afec2ddf322e-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:53.976077 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:53.976053 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fda6774-a044-441f-8357-afec2ddf322e-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:53.976268 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:53.976086 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fda6774-a044-441f-8357-afec2ddf322e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:53.976268 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:53.976102 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fda6774-a044-441f-8357-afec2ddf322e-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:53.976268 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:53.976150 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fda6774-a044-441f-8357-afec2ddf322e-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:54.076798 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.076762 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fda6774-a044-441f-8357-afec2ddf322e-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:54.076994 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.076817 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fda6774-a044-441f-8357-afec2ddf322e-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:54.076994 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.076836 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l524l\" (UniqueName: \"kubernetes.io/projected/4fda6774-a044-441f-8357-afec2ddf322e-kube-api-access-l524l\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:54.076994 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.076856 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fda6774-a044-441f-8357-afec2ddf322e-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:54.076994 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.076894 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fda6774-a044-441f-8357-afec2ddf322e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:54.076994 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.076917 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fda6774-a044-441f-8357-afec2ddf322e-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:54.076994 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.076941 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fda6774-a044-441f-8357-afec2ddf322e-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:54.076994 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.076972 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fda6774-a044-441f-8357-afec2ddf322e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:54.077357 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.076997 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fda6774-a044-441f-8357-afec2ddf322e-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:54.077357 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.077256 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fda6774-a044-441f-8357-afec2ddf322e-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:54.077357 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.077322 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fda6774-a044-441f-8357-afec2ddf322e-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:54.077502 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.077385 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fda6774-a044-441f-8357-afec2ddf322e-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:54.077502 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.077452 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fda6774-a044-441f-8357-afec2ddf322e-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:54.077848 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.077789 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fda6774-a044-441f-8357-afec2ddf322e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:54.079368 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.079348 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fda6774-a044-441f-8357-afec2ddf322e-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:54.079600 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.079583 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fda6774-a044-441f-8357-afec2ddf322e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:54.084236 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.084217 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l524l\" (UniqueName: \"kubernetes.io/projected/4fda6774-a044-441f-8357-afec2ddf322e-kube-api-access-l524l\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:54.084323 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.084299 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fda6774-a044-441f-8357-afec2ddf322e-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-b6v4x\" (UID: \"4fda6774-a044-441f-8357-afec2ddf322e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:54.238096 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.238009 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:54.368913 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.368888 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x"] Jul 16 05:01:54.370917 ip-10-0-135-187 kubenswrapper[2561]: W0716 05:01:54.370884 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fda6774_a044_441f_8357_afec2ddf322e.slice/crio-881401c3ff22a781e3b0c8fbcd8c58580125748537b780531c47c52a1b7c2286 WatchSource:0}: Error finding container 881401c3ff22a781e3b0c8fbcd8c58580125748537b780531c47c52a1b7c2286: Status 404 returned error can't find the container with id 881401c3ff22a781e3b0c8fbcd8c58580125748537b780531c47c52a1b7c2286 Jul 16 05:01:54.372947 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.372910 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:01:54.373072 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.372972 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:01:54.373072 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.372998 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:01:54.643283 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.643241 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" event={"ID":"4fda6774-a044-441f-8357-afec2ddf322e","Type":"ContainerStarted","Data":"dbd67fbf6b763119b16a36a0d5c1492d06abfac9de545608069bdbe1dea18279"} Jul 16 05:01:54.643463 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.643304 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" event={"ID":"4fda6774-a044-441f-8357-afec2ddf322e","Type":"ContainerStarted","Data":"881401c3ff22a781e3b0c8fbcd8c58580125748537b780531c47c52a1b7c2286"} Jul 16 05:01:54.664029 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:54.663983 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" podStartSLOduration=1.6639673419999998 podStartE2EDuration="1.663967342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:01:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:01:54.662354733 +0000 UTC m=+705.034477952" watchObservedRunningTime="2026-07-16 05:01:54.663967342 +0000 UTC m=+705.036090546" Jul 16 05:01:55.238465 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:55.238426 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:55.243328 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:55.243305 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:55.648816 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:55.648789 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:01:55.649625 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:01:55.649607 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:03:25.649962 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:25.649923 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-b6v4x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused" start-of-body= Jul 16 05:03:25.650349 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:25.649991 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" podUID="4fda6774-a044-441f-8357-afec2ddf322e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused" Jul 16 05:03:26.024289 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:26.024201 2561 generic.go:356] "Generic (PLEG): container finished" podID="4fda6774-a044-441f-8357-afec2ddf322e" containerID="dbd67fbf6b763119b16a36a0d5c1492d06abfac9de545608069bdbe1dea18279" exitCode=0 Jul 16 05:03:26.024289 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:26.024251 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" event={"ID":"4fda6774-a044-441f-8357-afec2ddf322e","Type":"ContainerDied","Data":"dbd67fbf6b763119b16a36a0d5c1492d06abfac9de545608069bdbe1dea18279"} Jul 16 05:03:26.024753 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:26.024726 2561 scope.go:117] "RemoveContainer" containerID="dbd67fbf6b763119b16a36a0d5c1492d06abfac9de545608069bdbe1dea18279" Jul 16 05:03:26.025847 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:26.025820 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:03:26.025955 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:26.025884 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:03:26.025955 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:26.025913 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:03:27.030422 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:27.030390 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" event={"ID":"4fda6774-a044-441f-8357-afec2ddf322e","Type":"ContainerStarted","Data":"24330854f7671363e9d348be685bd0e09e739a32e59cc0c798c3d95832e2f078"} Jul 16 05:03:27.238649 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:27.238605 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:03:27.238871 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:27.238847 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-b6v4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused" start-of-body= Jul 16 05:03:27.238917 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:27.238899 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" podUID="4fda6774-a044-441f-8357-afec2ddf322e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused" Jul 16 05:03:28.238677 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:28.238635 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-b6v4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused" start-of-body= Jul 16 05:03:28.239108 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:28.238721 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" podUID="4fda6774-a044-441f-8357-afec2ddf322e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused" Jul 16 05:03:29.239344 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:29.239308 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-b6v4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused" start-of-body= Jul 16 05:03:29.239783 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:29.239362 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" podUID="4fda6774-a044-441f-8357-afec2ddf322e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused" Jul 16 05:03:30.238765 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:30.238732 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-b6v4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused" start-of-body= Jul 16 05:03:30.238933 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:30.238781 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" podUID="4fda6774-a044-441f-8357-afec2ddf322e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused" Jul 16 05:03:31.242256 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:31.242220 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:03:31.242619 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:31.242505 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:03:31.243131 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:31.243111 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-b6v4x" Jul 16 05:03:52.226744 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.226711 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w"] Jul 16 05:03:52.229051 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.229024 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" Jul 16 05:03:52.232831 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.232810 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 16 05:03:52.232947 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.232812 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-qtz5t\"" Jul 16 05:03:52.232947 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.232819 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 16 05:03:52.232947 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.232936 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 16 05:03:52.238496 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.238476 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w"] Jul 16 05:03:52.279854 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.279829 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9450e32b-d341-499e-8142-49addd0af661-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z7n8w\" (UID: \"9450e32b-d341-499e-8142-49addd0af661\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" Jul 16 05:03:52.279995 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.279875 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9450e32b-d341-499e-8142-49addd0af661-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z7n8w\" (UID: \"9450e32b-d341-499e-8142-49addd0af661\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" Jul 16 05:03:52.279995 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.279924 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9450e32b-d341-499e-8142-49addd0af661-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z7n8w\" (UID: \"9450e32b-d341-499e-8142-49addd0af661\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" Jul 16 05:03:52.279995 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.279954 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhtmx\" (UniqueName: \"kubernetes.io/projected/9450e32b-d341-499e-8142-49addd0af661-kube-api-access-fhtmx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z7n8w\" (UID: \"9450e32b-d341-499e-8142-49addd0af661\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" Jul 16 05:03:52.280100 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.280021 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9450e32b-d341-499e-8142-49addd0af661-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z7n8w\" (UID: \"9450e32b-d341-499e-8142-49addd0af661\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" Jul 16 05:03:52.280100 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.280059 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9450e32b-d341-499e-8142-49addd0af661-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z7n8w\" (UID: \"9450e32b-d341-499e-8142-49addd0af661\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" Jul 16 05:03:52.381289 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.381254 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9450e32b-d341-499e-8142-49addd0af661-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z7n8w\" (UID: \"9450e32b-d341-499e-8142-49addd0af661\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" Jul 16 05:03:52.381509 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.381301 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9450e32b-d341-499e-8142-49addd0af661-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z7n8w\" (UID: \"9450e32b-d341-499e-8142-49addd0af661\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" Jul 16 05:03:52.381509 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.381342 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9450e32b-d341-499e-8142-49addd0af661-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z7n8w\" (UID: \"9450e32b-d341-499e-8142-49addd0af661\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" Jul 16 05:03:52.381509 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.381366 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9450e32b-d341-499e-8142-49addd0af661-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z7n8w\" (UID: \"9450e32b-d341-499e-8142-49addd0af661\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" Jul 16 05:03:52.381509 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.381383 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhtmx\" (UniqueName: \"kubernetes.io/projected/9450e32b-d341-499e-8142-49addd0af661-kube-api-access-fhtmx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z7n8w\" (UID: \"9450e32b-d341-499e-8142-49addd0af661\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" Jul 16 05:03:52.381751 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.381521 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9450e32b-d341-499e-8142-49addd0af661-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z7n8w\" (UID: \"9450e32b-d341-499e-8142-49addd0af661\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" Jul 16 05:03:52.381751 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.381732 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9450e32b-d341-499e-8142-49addd0af661-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z7n8w\" (UID: \"9450e32b-d341-499e-8142-49addd0af661\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" Jul 16 05:03:52.381846 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.381781 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9450e32b-d341-499e-8142-49addd0af661-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z7n8w\" (UID: \"9450e32b-d341-499e-8142-49addd0af661\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" Jul 16 05:03:52.381882 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.381853 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9450e32b-d341-499e-8142-49addd0af661-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z7n8w\" (UID: \"9450e32b-d341-499e-8142-49addd0af661\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" Jul 16 05:03:52.383827 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.383806 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9450e32b-d341-499e-8142-49addd0af661-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z7n8w\" (UID: \"9450e32b-d341-499e-8142-49addd0af661\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" Jul 16 05:03:52.384027 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.384008 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9450e32b-d341-499e-8142-49addd0af661-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z7n8w\" (UID: \"9450e32b-d341-499e-8142-49addd0af661\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" Jul 16 05:03:52.389390 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.389366 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhtmx\" (UniqueName: \"kubernetes.io/projected/9450e32b-d341-499e-8142-49addd0af661-kube-api-access-fhtmx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z7n8w\" (UID: \"9450e32b-d341-499e-8142-49addd0af661\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" Jul 16 05:03:52.539978 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.539953 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" Jul 16 05:03:52.664100 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.664073 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w"] Jul 16 05:03:52.665720 ip-10-0-135-187 kubenswrapper[2561]: W0716 05:03:52.665677 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9450e32b_d341_499e_8142_49addd0af661.slice/crio-fc54d2dc60ec4067d18a064b9535c7fbf78c24cbacd2cc2680f763e37192b775 WatchSource:0}: Error finding container fc54d2dc60ec4067d18a064b9535c7fbf78c24cbacd2cc2680f763e37192b775: Status 404 returned error can't find the container with id fc54d2dc60ec4067d18a064b9535c7fbf78c24cbacd2cc2680f763e37192b775 Jul 16 05:03:52.667179 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:52.667157 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 05:03:53.138520 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:53.138473 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" event={"ID":"9450e32b-d341-499e-8142-49addd0af661","Type":"ContainerStarted","Data":"fc54d2dc60ec4067d18a064b9535c7fbf78c24cbacd2cc2680f763e37192b775"} Jul 16 05:03:54.144338 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:54.144302 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" event={"ID":"9450e32b-d341-499e-8142-49addd0af661","Type":"ContainerStarted","Data":"2ffc77d754967941007fee2957dfd66ae6ed039a3cee62f17388b52156567a46"} Jul 16 05:03:54.160207 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:03:54.160158 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" podStartSLOduration=1.144539167 podStartE2EDuration="2.16014354s" totalImagesPullingTime="1.015604373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:03:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:03:54.159281474 +0000 UTC m=+824.531404698" watchObservedRunningTime="2026-07-16 05:03:54.16014354 +0000 UTC m=+824.532266795" Jul 16 05:04:02.540582 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:02.540538 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" Jul 16 05:04:02.540994 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:02.540615 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" Jul 16 05:04:02.552884 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:02.552857 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" Jul 16 05:04:03.191738 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:03.191705 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z7n8w" Jul 16 05:04:20.619387 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:20.619351 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9"] Jul 16 05:04:20.626084 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:20.626062 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" Jul 16 05:04:20.628670 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:20.628648 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 16 05:04:20.633048 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:20.632962 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9"] Jul 16 05:04:20.729972 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:20.729942 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df804050-3468-47a9-be52-161b28b87345-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cjdx9\" (UID: \"df804050-3468-47a9-be52-161b28b87345\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" Jul 16 05:04:20.730162 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:20.729990 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df804050-3468-47a9-be52-161b28b87345-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cjdx9\" (UID: \"df804050-3468-47a9-be52-161b28b87345\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" Jul 16 05:04:20.730162 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:20.730017 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df804050-3468-47a9-be52-161b28b87345-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cjdx9\" (UID: \"df804050-3468-47a9-be52-161b28b87345\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" Jul 16 05:04:20.730162 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:20.730070 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df804050-3468-47a9-be52-161b28b87345-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cjdx9\" (UID: \"df804050-3468-47a9-be52-161b28b87345\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" Jul 16 05:04:20.730162 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:20.730110 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxkkn\" (UniqueName: \"kubernetes.io/projected/df804050-3468-47a9-be52-161b28b87345-kube-api-access-wxkkn\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cjdx9\" (UID: \"df804050-3468-47a9-be52-161b28b87345\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" Jul 16 05:04:20.730310 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:20.730170 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df804050-3468-47a9-be52-161b28b87345-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cjdx9\" (UID: \"df804050-3468-47a9-be52-161b28b87345\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" Jul 16 05:04:20.831069 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:20.831031 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df804050-3468-47a9-be52-161b28b87345-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cjdx9\" (UID: \"df804050-3468-47a9-be52-161b28b87345\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" Jul 16 05:04:20.831248 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:20.831083 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df804050-3468-47a9-be52-161b28b87345-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cjdx9\" (UID: \"df804050-3468-47a9-be52-161b28b87345\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" Jul 16 05:04:20.831248 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:20.831121 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df804050-3468-47a9-be52-161b28b87345-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cjdx9\" (UID: \"df804050-3468-47a9-be52-161b28b87345\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" Jul 16 05:04:20.831248 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:20.831145 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxkkn\" (UniqueName: \"kubernetes.io/projected/df804050-3468-47a9-be52-161b28b87345-kube-api-access-wxkkn\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cjdx9\" (UID: \"df804050-3468-47a9-be52-161b28b87345\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" Jul 16 05:04:20.831248 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:20.831181 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df804050-3468-47a9-be52-161b28b87345-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cjdx9\" (UID: \"df804050-3468-47a9-be52-161b28b87345\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" Jul 16 05:04:20.831449 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:20.831253 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df804050-3468-47a9-be52-161b28b87345-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cjdx9\" (UID: \"df804050-3468-47a9-be52-161b28b87345\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" Jul 16 05:04:20.831560 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:20.831522 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df804050-3468-47a9-be52-161b28b87345-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cjdx9\" (UID: \"df804050-3468-47a9-be52-161b28b87345\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" Jul 16 05:04:20.831560 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:20.831556 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df804050-3468-47a9-be52-161b28b87345-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cjdx9\" (UID: \"df804050-3468-47a9-be52-161b28b87345\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" Jul 16 05:04:20.831741 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:20.831593 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df804050-3468-47a9-be52-161b28b87345-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cjdx9\" (UID: \"df804050-3468-47a9-be52-161b28b87345\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" Jul 16 05:04:20.833542 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:20.833524 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df804050-3468-47a9-be52-161b28b87345-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cjdx9\" (UID: \"df804050-3468-47a9-be52-161b28b87345\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" Jul 16 05:04:20.833850 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:20.833833 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df804050-3468-47a9-be52-161b28b87345-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cjdx9\" (UID: \"df804050-3468-47a9-be52-161b28b87345\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" Jul 16 05:04:20.839021 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:20.838997 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxkkn\" (UniqueName: \"kubernetes.io/projected/df804050-3468-47a9-be52-161b28b87345-kube-api-access-wxkkn\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cjdx9\" (UID: \"df804050-3468-47a9-be52-161b28b87345\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" Jul 16 05:04:20.937065 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:20.936975 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" Jul 16 05:04:21.066201 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:21.066176 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9"] Jul 16 05:04:21.068184 ip-10-0-135-187 kubenswrapper[2561]: W0716 05:04:21.068156 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf804050_3468_47a9_be52_161b28b87345.slice/crio-3bc165137db27798a55c8460dfa453ba5d14d31c2b8ee835c6b6093dcdc96435 WatchSource:0}: Error finding container 3bc165137db27798a55c8460dfa453ba5d14d31c2b8ee835c6b6093dcdc96435: Status 404 returned error can't find the container with id 3bc165137db27798a55c8460dfa453ba5d14d31c2b8ee835c6b6093dcdc96435 Jul 16 05:04:21.256508 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:21.256471 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" event={"ID":"df804050-3468-47a9-be52-161b28b87345","Type":"ContainerStarted","Data":"3bc165137db27798a55c8460dfa453ba5d14d31c2b8ee835c6b6093dcdc96435"} Jul 16 05:04:22.261794 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:22.261754 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" event={"ID":"df804050-3468-47a9-be52-161b28b87345","Type":"ContainerStarted","Data":"da41fc876ca5764ce946a832912aa1bc13f8240514d1c96cf81a31e86cac99a1"} Jul 16 05:04:22.278250 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:22.278201 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" podStartSLOduration=2.100808182 podStartE2EDuration="2.278186387s" totalImagesPullingTime="177.378205ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:04:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:04:22.277633258 +0000 UTC m=+852.649756487" watchObservedRunningTime="2026-07-16 05:04:22.278186387 +0000 UTC m=+852.650309636" Jul 16 05:04:30.937289 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:30.937246 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" Jul 16 05:04:30.937828 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:30.937333 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" Jul 16 05:04:30.949589 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:30.949562 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" Jul 16 05:04:31.312031 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:04:31.312002 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cjdx9" Jul 16 05:19:17.324407 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.324375 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2"] Jul 16 05:19:17.327578 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.327561 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.330327 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.330304 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f2ca692b-openshift-default-dockercfg-ck6gp\"" Jul 16 05:19:17.345593 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.345568 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2"] Jul 16 05:19:17.381806 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.381777 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-credential-socket\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.381931 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.381810 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-workload-socket\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.381931 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.381831 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istio-envoy\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.381931 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.381849 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istiod-ca-cert\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.381931 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.381906 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl7nt\" (UniqueName: \"kubernetes.io/projected/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-kube-api-access-bl7nt\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.382064 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.381935 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istio-data\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.382064 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.381969 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istio-token\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.382064 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.382011 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-workload-certs\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.382152 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.382070 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istio-podinfo\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.482963 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.482920 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istio-podinfo\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.482963 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.482967 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-credential-socket\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.483195 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.483082 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-workload-socket\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.483195 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.483124 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istio-envoy\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.483195 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.483144 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istiod-ca-cert\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.483195 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.483190 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl7nt\" (UniqueName: \"kubernetes.io/projected/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-kube-api-access-bl7nt\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.483415 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.483223 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istio-data\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.483415 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.483242 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istio-token\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.483415 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.483378 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-workload-certs\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.483415 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.483394 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-credential-socket\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.483711 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.483415 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-workload-socket\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.483778 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.483720 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istio-data\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.483995 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.483773 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-workload-certs\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.484258 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.484229 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istiod-ca-cert\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.485564 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.485541 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istio-envoy\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.485708 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.485673 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istio-podinfo\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.492151 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.492131 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl7nt\" (UniqueName: \"kubernetes.io/projected/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-kube-api-access-bl7nt\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.492532 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.492513 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istio-token\") pod \"e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.638396 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.638308 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:17.772280 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.772254 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2"] Jul 16 05:19:17.774023 ip-10-0-135-187 kubenswrapper[2561]: W0716 05:19:17.773989 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92ca8168_8aa4_4d16_b30c_f65ebb217a9c.slice/crio-831f0f280aa85cc285d8ed4029caa6fecfbc353fba11c0b50493f5842dc6d96b WatchSource:0}: Error finding container 831f0f280aa85cc285d8ed4029caa6fecfbc353fba11c0b50493f5842dc6d96b: Status 404 returned error can't find the container with id 831f0f280aa85cc285d8ed4029caa6fecfbc353fba11c0b50493f5842dc6d96b Jul 16 05:19:17.775984 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.775956 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:19:17.776075 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.776042 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:19:17.776075 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:17.776070 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:19:18.063364 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:18.063325 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" event={"ID":"92ca8168-8aa4-4d16-b30c-f65ebb217a9c","Type":"ContainerStarted","Data":"aa4cae34df6805a5ac00ff0fd27343a4247537b315002c26bbce3b364e7dcfa9"} Jul 16 05:19:18.063571 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:18.063396 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" event={"ID":"92ca8168-8aa4-4d16-b30c-f65ebb217a9c","Type":"ContainerStarted","Data":"831f0f280aa85cc285d8ed4029caa6fecfbc353fba11c0b50493f5842dc6d96b"} Jul 16 05:19:18.081966 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:18.081921 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" podStartSLOduration=1.081907615 podStartE2EDuration="1.081907615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:19:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:19:18.080143089 +0000 UTC m=+1748.452266325" watchObservedRunningTime="2026-07-16 05:19:18.081907615 +0000 UTC m=+1748.454030819" Jul 16 05:19:18.638879 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:18.638839 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:18.660174 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:18.660125 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:19.068155 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:19.068127 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:19.069195 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:19.069174 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:30.118598 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:30.118560 2561 generic.go:356] "Generic (PLEG): container finished" podID="92ca8168-8aa4-4d16-b30c-f65ebb217a9c" containerID="aa4cae34df6805a5ac00ff0fd27343a4247537b315002c26bbce3b364e7dcfa9" exitCode=0 Jul 16 05:19:30.119033 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:30.118634 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" event={"ID":"92ca8168-8aa4-4d16-b30c-f65ebb217a9c","Type":"ContainerDied","Data":"aa4cae34df6805a5ac00ff0fd27343a4247537b315002c26bbce3b364e7dcfa9"} Jul 16 05:19:30.119082 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:30.119042 2561 scope.go:117] "RemoveContainer" containerID="aa4cae34df6805a5ac00ff0fd27343a4247537b315002c26bbce3b364e7dcfa9" Jul 16 05:19:30.120065 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:30.120037 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:19:30.120138 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:30.120121 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:19:30.120186 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:30.120171 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:19:31.124875 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:31.124841 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" event={"ID":"92ca8168-8aa4-4d16-b30c-f65ebb217a9c","Type":"ContainerStarted","Data":"273c6feb4be88b47a6f168f1c0c4818b7bdd17ce20ebb52da13ec86a6fc461e8"} Jul 16 05:19:31.638774 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:31.638734 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:31.639614 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:31.639590 2561 patch_prober.go:28] interesting pod/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" start-of-body= Jul 16 05:19:31.639666 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:31.639642 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" podUID="92ca8168-8aa4-4d16-b30c-f65ebb217a9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" Jul 16 05:19:32.639653 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:32.639615 2561 patch_prober.go:28] interesting pod/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" start-of-body= Jul 16 05:19:32.640082 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:32.639675 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" podUID="92ca8168-8aa4-4d16-b30c-f65ebb217a9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" Jul 16 05:19:33.638915 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:33.638871 2561 patch_prober.go:28] interesting pod/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" start-of-body= Jul 16 05:19:33.639097 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:33.638944 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" podUID="92ca8168-8aa4-4d16-b30c-f65ebb217a9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" Jul 16 05:19:34.077862 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:34.077809 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:34.639099 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:34.639067 2561 patch_prober.go:28] interesting pod/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" start-of-body= Jul 16 05:19:34.639276 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:34.639125 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" podUID="92ca8168-8aa4-4d16-b30c-f65ebb217a9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" Jul 16 05:19:35.642836 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:35.642806 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:19:35.643634 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:19:35.643618 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:20:48.916731 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:48.916665 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2"] Jul 16 05:20:48.917262 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:48.916922 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" podUID="92ca8168-8aa4-4d16-b30c-f65ebb217a9c" containerName="istio-proxy" containerID="cri-o://273c6feb4be88b47a6f168f1c0c4818b7bdd17ce20ebb52da13ec86a6fc461e8" gracePeriod=30 Jul 16 05:20:49.573699 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.573655 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j"] Jul 16 05:20:49.577282 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.577259 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.580647 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.580619 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-1eae9b50-gw-openshift-default-dockercfg-8kd7x\"" Jul 16 05:20:49.588546 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.588386 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-workload-certs\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.588546 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.588439 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istio-podinfo\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.588546 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.588467 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istiod-ca-cert\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.588546 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.588494 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istio-data\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.588858 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.588612 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-workload-socket\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.588858 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.588645 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istio-token\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.588858 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.588678 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-credential-socket\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.588858 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.588730 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istio-envoy\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.588858 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.588757 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vkzl\" (UniqueName: \"kubernetes.io/projected/ba7c5dbb-e277-4480-8222-dd5a372a72f5-kube-api-access-4vkzl\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.589480 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.589460 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j"] Jul 16 05:20:49.689293 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.689252 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-workload-certs\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.689471 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.689305 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istio-podinfo\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.689471 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.689335 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istiod-ca-cert\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.689471 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.689362 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istio-data\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.689471 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.689405 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-workload-socket\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.689471 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.689432 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istio-token\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.689471 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.689466 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-credential-socket\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.689810 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.689494 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istio-envoy\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.689869 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.689852 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-workload-socket\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.689936 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.689910 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vkzl\" (UniqueName: \"kubernetes.io/projected/ba7c5dbb-e277-4480-8222-dd5a372a72f5-kube-api-access-4vkzl\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.690255 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.690198 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istiod-ca-cert\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.690368 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.690334 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istio-data\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.690425 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.690381 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-credential-socket\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.690551 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.690527 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-workload-certs\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.692825 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.692803 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istio-envoy\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.693280 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.693222 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istio-podinfo\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.704949 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.702863 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vkzl\" (UniqueName: \"kubernetes.io/projected/ba7c5dbb-e277-4480-8222-dd5a372a72f5-kube-api-access-4vkzl\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.704949 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.704739 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istio-token\") pod \"e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:49.889843 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:49.889750 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:50.039049 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:50.038977 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j"] Jul 16 05:20:50.041730 ip-10-0-135-187 kubenswrapper[2561]: W0716 05:20:50.041676 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba7c5dbb_e277_4480_8222_dd5a372a72f5.slice/crio-f3f183030c3986782eca194c934bfd6494d18d506056896b8745d94fb2dc4455 WatchSource:0}: Error finding container f3f183030c3986782eca194c934bfd6494d18d506056896b8745d94fb2dc4455: Status 404 returned error can't find the container with id f3f183030c3986782eca194c934bfd6494d18d506056896b8745d94fb2dc4455 Jul 16 05:20:50.043964 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:50.043934 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:20:50.044029 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:50.043999 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:20:50.044029 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:50.044026 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:20:50.472802 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:50.472758 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" event={"ID":"ba7c5dbb-e277-4480-8222-dd5a372a72f5","Type":"ContainerStarted","Data":"2eef8b43bcc782aa8aebdac0d21b4f8f03c3ed47107aa3b1fc02bb1a99f9a07b"} Jul 16 05:20:50.473010 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:50.472861 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" event={"ID":"ba7c5dbb-e277-4480-8222-dd5a372a72f5","Type":"ContainerStarted","Data":"f3f183030c3986782eca194c934bfd6494d18d506056896b8745d94fb2dc4455"} Jul 16 05:20:50.495337 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:50.495279 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" podStartSLOduration=1.495260654 podStartE2EDuration="1.495260654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:20:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:20:50.493408232 +0000 UTC m=+1840.865531466" watchObservedRunningTime="2026-07-16 05:20:50.495260654 +0000 UTC m=+1840.867383866" Jul 16 05:20:50.646079 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:50.646040 2561 patch_prober.go:28] interesting pod/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 05:20:50.646231 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:50.646110 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" podUID="92ca8168-8aa4-4d16-b30c-f65ebb217a9c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 05:20:50.889997 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:50.889957 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:51.895209 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:51.895179 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:52.481171 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:52.481135 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:52.482139 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:52.482118 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:20:54.164183 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.164159 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:20:54.223187 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.223148 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bl7nt\" (UniqueName: \"kubernetes.io/projected/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-kube-api-access-bl7nt\") pod \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " Jul 16 05:20:54.223187 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.223190 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istiod-ca-cert\") pod \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " Jul 16 05:20:54.223427 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.223256 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-credential-socket\") pod \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " Jul 16 05:20:54.223427 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.223271 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istio-token\") pod \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " Jul 16 05:20:54.223427 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.223303 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-workload-certs\") pod \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " Jul 16 05:20:54.223427 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.223337 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istio-envoy\") pod \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " Jul 16 05:20:54.223427 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.223361 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-workload-socket\") pod \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " Jul 16 05:20:54.223427 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.223396 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istio-podinfo\") pod \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " Jul 16 05:20:54.223721 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.223437 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istio-data\") pod \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\" (UID: \"92ca8168-8aa4-4d16-b30c-f65ebb217a9c\") " Jul 16 05:20:54.223721 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.223612 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "92ca8168-8aa4-4d16-b30c-f65ebb217a9c" (UID: "92ca8168-8aa4-4d16-b30c-f65ebb217a9c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:20:54.223821 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.223755 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-credential-socket\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:20:54.223821 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.223786 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "92ca8168-8aa4-4d16-b30c-f65ebb217a9c" (UID: "92ca8168-8aa4-4d16-b30c-f65ebb217a9c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:20:54.223924 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.223892 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "92ca8168-8aa4-4d16-b30c-f65ebb217a9c" (UID: "92ca8168-8aa4-4d16-b30c-f65ebb217a9c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:20:54.224003 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.223977 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "92ca8168-8aa4-4d16-b30c-f65ebb217a9c" (UID: "92ca8168-8aa4-4d16-b30c-f65ebb217a9c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 05:20:54.224519 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.224493 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istio-data" (OuterVolumeSpecName: "istio-data") pod "92ca8168-8aa4-4d16-b30c-f65ebb217a9c" (UID: "92ca8168-8aa4-4d16-b30c-f65ebb217a9c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:20:54.226506 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.226476 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "92ca8168-8aa4-4d16-b30c-f65ebb217a9c" (UID: "92ca8168-8aa4-4d16-b30c-f65ebb217a9c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:20:54.226506 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.226486 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-kube-api-access-bl7nt" (OuterVolumeSpecName: "kube-api-access-bl7nt") pod "92ca8168-8aa4-4d16-b30c-f65ebb217a9c" (UID: "92ca8168-8aa4-4d16-b30c-f65ebb217a9c"). InnerVolumeSpecName "kube-api-access-bl7nt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:20:54.226676 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.226529 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istio-token" (OuterVolumeSpecName: "istio-token") pod "92ca8168-8aa4-4d16-b30c-f65ebb217a9c" (UID: "92ca8168-8aa4-4d16-b30c-f65ebb217a9c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:20:54.226676 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.226572 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "92ca8168-8aa4-4d16-b30c-f65ebb217a9c" (UID: "92ca8168-8aa4-4d16-b30c-f65ebb217a9c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 05:20:54.324340 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.324296 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-workload-certs\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:20:54.324340 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.324333 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istio-envoy\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:20:54.324340 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.324342 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-workload-socket\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:20:54.324340 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.324353 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istio-podinfo\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:20:54.324594 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.324361 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istio-data\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:20:54.324594 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.324369 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bl7nt\" (UniqueName: \"kubernetes.io/projected/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-kube-api-access-bl7nt\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:20:54.324594 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.324380 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istiod-ca-cert\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:20:54.324594 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.324389 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92ca8168-8aa4-4d16-b30c-f65ebb217a9c-istio-token\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:20:54.491052 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.490968 2561 generic.go:356] "Generic (PLEG): container finished" podID="92ca8168-8aa4-4d16-b30c-f65ebb217a9c" containerID="273c6feb4be88b47a6f168f1c0c4818b7bdd17ce20ebb52da13ec86a6fc461e8" exitCode=0 Jul 16 05:20:54.491052 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.491027 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" Jul 16 05:20:54.491232 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.491064 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" event={"ID":"92ca8168-8aa4-4d16-b30c-f65ebb217a9c","Type":"ContainerDied","Data":"273c6feb4be88b47a6f168f1c0c4818b7bdd17ce20ebb52da13ec86a6fc461e8"} Jul 16 05:20:54.491232 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.491121 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2" event={"ID":"92ca8168-8aa4-4d16-b30c-f65ebb217a9c","Type":"ContainerDied","Data":"831f0f280aa85cc285d8ed4029caa6fecfbc353fba11c0b50493f5842dc6d96b"} Jul 16 05:20:54.491232 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.491153 2561 scope.go:117] "RemoveContainer" containerID="273c6feb4be88b47a6f168f1c0c4818b7bdd17ce20ebb52da13ec86a6fc461e8" Jul 16 05:20:54.501109 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.501083 2561 scope.go:117] "RemoveContainer" containerID="aa4cae34df6805a5ac00ff0fd27343a4247537b315002c26bbce3b364e7dcfa9" Jul 16 05:20:54.508740 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.508721 2561 scope.go:117] "RemoveContainer" containerID="273c6feb4be88b47a6f168f1c0c4818b7bdd17ce20ebb52da13ec86a6fc461e8" Jul 16 05:20:54.509009 ip-10-0-135-187 kubenswrapper[2561]: E0716 05:20:54.508988 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"273c6feb4be88b47a6f168f1c0c4818b7bdd17ce20ebb52da13ec86a6fc461e8\": container with ID starting with 273c6feb4be88b47a6f168f1c0c4818b7bdd17ce20ebb52da13ec86a6fc461e8 not found: ID does not exist" containerID="273c6feb4be88b47a6f168f1c0c4818b7bdd17ce20ebb52da13ec86a6fc461e8" Jul 16 05:20:54.509109 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.509015 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"273c6feb4be88b47a6f168f1c0c4818b7bdd17ce20ebb52da13ec86a6fc461e8"} err="failed to get container status \"273c6feb4be88b47a6f168f1c0c4818b7bdd17ce20ebb52da13ec86a6fc461e8\": rpc error: code = NotFound desc = could not find container \"273c6feb4be88b47a6f168f1c0c4818b7bdd17ce20ebb52da13ec86a6fc461e8\": container with ID starting with 273c6feb4be88b47a6f168f1c0c4818b7bdd17ce20ebb52da13ec86a6fc461e8 not found: ID does not exist" Jul 16 05:20:54.509109 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.509034 2561 scope.go:117] "RemoveContainer" containerID="aa4cae34df6805a5ac00ff0fd27343a4247537b315002c26bbce3b364e7dcfa9" Jul 16 05:20:54.509253 ip-10-0-135-187 kubenswrapper[2561]: E0716 05:20:54.509237 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa4cae34df6805a5ac00ff0fd27343a4247537b315002c26bbce3b364e7dcfa9\": container with ID starting with aa4cae34df6805a5ac00ff0fd27343a4247537b315002c26bbce3b364e7dcfa9 not found: ID does not exist" containerID="aa4cae34df6805a5ac00ff0fd27343a4247537b315002c26bbce3b364e7dcfa9" Jul 16 05:20:54.509297 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.509257 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa4cae34df6805a5ac00ff0fd27343a4247537b315002c26bbce3b364e7dcfa9"} err="failed to get container status \"aa4cae34df6805a5ac00ff0fd27343a4247537b315002c26bbce3b364e7dcfa9\": rpc error: code = NotFound desc = could not find container \"aa4cae34df6805a5ac00ff0fd27343a4247537b315002c26bbce3b364e7dcfa9\": container with ID starting with aa4cae34df6805a5ac00ff0fd27343a4247537b315002c26bbce3b364e7dcfa9 not found: ID does not exist" Jul 16 05:20:54.513704 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.513660 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2"] Jul 16 05:20:54.515538 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:54.515519 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f2ca692b-openshift-default-58f6dd74f7-k8kp2"] Jul 16 05:20:56.105294 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:20:56.105256 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92ca8168-8aa4-4d16-b30c-f65ebb217a9c" path="/var/lib/kubelet/pods/92ca8168-8aa4-4d16-b30c-f65ebb217a9c/volumes" Jul 16 05:21:07.482749 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:21:07.482652 2561 patch_prober.go:28] interesting pod/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Jul 16 05:21:07.483195 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:21:07.482741 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" podUID="ba7c5dbb-e277-4480-8222-dd5a372a72f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Jul 16 05:21:07.549228 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:21:07.549196 2561 generic.go:356] "Generic (PLEG): container finished" podID="ba7c5dbb-e277-4480-8222-dd5a372a72f5" containerID="2eef8b43bcc782aa8aebdac0d21b4f8f03c3ed47107aa3b1fc02bb1a99f9a07b" exitCode=0 Jul 16 05:21:07.549418 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:21:07.549275 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" event={"ID":"ba7c5dbb-e277-4480-8222-dd5a372a72f5","Type":"ContainerDied","Data":"2eef8b43bcc782aa8aebdac0d21b4f8f03c3ed47107aa3b1fc02bb1a99f9a07b"} Jul 16 05:21:07.549765 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:21:07.549747 2561 scope.go:117] "RemoveContainer" containerID="2eef8b43bcc782aa8aebdac0d21b4f8f03c3ed47107aa3b1fc02bb1a99f9a07b" Jul 16 05:21:07.550717 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:21:07.550676 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:21:07.550812 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:21:07.550771 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:21:07.550812 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:21:07.550806 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:21:08.558478 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:21:08.558436 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" event={"ID":"ba7c5dbb-e277-4480-8222-dd5a372a72f5","Type":"ContainerStarted","Data":"e81414355aede4ebd2767fed4987f8a0e32607dfeac5b55591dd5d354e8c040f"} Jul 16 05:21:08.890871 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:21:08.890763 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:21:08.891021 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:21:08.890991 2561 patch_prober.go:28] interesting pod/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Jul 16 05:21:08.891066 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:21:08.891042 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" podUID="ba7c5dbb-e277-4480-8222-dd5a372a72f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Jul 16 05:21:09.891042 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:21:09.891001 2561 patch_prober.go:28] interesting pod/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Jul 16 05:21:09.891443 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:21:09.891077 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" podUID="ba7c5dbb-e277-4480-8222-dd5a372a72f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Jul 16 05:21:10.891064 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:21:10.891021 2561 patch_prober.go:28] interesting pod/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Jul 16 05:21:10.891632 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:21:10.891098 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" podUID="ba7c5dbb-e277-4480-8222-dd5a372a72f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Jul 16 05:21:11.891313 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:21:11.891268 2561 patch_prober.go:28] interesting pod/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Jul 16 05:21:11.891736 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:21:11.891331 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" podUID="ba7c5dbb-e277-4480-8222-dd5a372a72f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Jul 16 05:21:12.894438 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:21:12.894409 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:21:12.894867 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:21:12.894707 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:21:12.895405 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:21:12.895387 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:21:55.498856 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:21:55.498774 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j"] Jul 16 05:21:55.499300 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:21:55.499047 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" podUID="ba7c5dbb-e277-4480-8222-dd5a372a72f5" containerName="istio-proxy" containerID="cri-o://e81414355aede4ebd2767fed4987f8a0e32607dfeac5b55591dd5d354e8c040f" gracePeriod=30 Jul 16 05:21:57.897348 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:21:57.897313 2561 patch_prober.go:28] interesting pod/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 16 05:21:57.897793 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:21:57.897373 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" podUID="ba7c5dbb-e277-4480-8222-dd5a372a72f5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 05:22:00.736705 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.736658 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:22:00.788472 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.788439 2561 generic.go:356] "Generic (PLEG): container finished" podID="ba7c5dbb-e277-4480-8222-dd5a372a72f5" containerID="e81414355aede4ebd2767fed4987f8a0e32607dfeac5b55591dd5d354e8c040f" exitCode=0 Jul 16 05:22:00.788595 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.788495 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" Jul 16 05:22:00.788595 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.788532 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" event={"ID":"ba7c5dbb-e277-4480-8222-dd5a372a72f5","Type":"ContainerDied","Data":"e81414355aede4ebd2767fed4987f8a0e32607dfeac5b55591dd5d354e8c040f"} Jul 16 05:22:00.788696 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.788634 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j" event={"ID":"ba7c5dbb-e277-4480-8222-dd5a372a72f5","Type":"ContainerDied","Data":"f3f183030c3986782eca194c934bfd6494d18d506056896b8745d94fb2dc4455"} Jul 16 05:22:00.788696 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.788668 2561 scope.go:117] "RemoveContainer" containerID="e81414355aede4ebd2767fed4987f8a0e32607dfeac5b55591dd5d354e8c040f" Jul 16 05:22:00.797632 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.797609 2561 scope.go:117] "RemoveContainer" containerID="2eef8b43bcc782aa8aebdac0d21b4f8f03c3ed47107aa3b1fc02bb1a99f9a07b" Jul 16 05:22:00.806594 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.806573 2561 scope.go:117] "RemoveContainer" containerID="e81414355aede4ebd2767fed4987f8a0e32607dfeac5b55591dd5d354e8c040f" Jul 16 05:22:00.806899 ip-10-0-135-187 kubenswrapper[2561]: E0716 05:22:00.806880 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e81414355aede4ebd2767fed4987f8a0e32607dfeac5b55591dd5d354e8c040f\": container with ID starting with e81414355aede4ebd2767fed4987f8a0e32607dfeac5b55591dd5d354e8c040f not found: ID does not exist" containerID="e81414355aede4ebd2767fed4987f8a0e32607dfeac5b55591dd5d354e8c040f" Jul 16 05:22:00.806962 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.806911 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e81414355aede4ebd2767fed4987f8a0e32607dfeac5b55591dd5d354e8c040f"} err="failed to get container status \"e81414355aede4ebd2767fed4987f8a0e32607dfeac5b55591dd5d354e8c040f\": rpc error: code = NotFound desc = could not find container \"e81414355aede4ebd2767fed4987f8a0e32607dfeac5b55591dd5d354e8c040f\": container with ID starting with e81414355aede4ebd2767fed4987f8a0e32607dfeac5b55591dd5d354e8c040f not found: ID does not exist" Jul 16 05:22:00.806962 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.806929 2561 scope.go:117] "RemoveContainer" containerID="2eef8b43bcc782aa8aebdac0d21b4f8f03c3ed47107aa3b1fc02bb1a99f9a07b" Jul 16 05:22:00.807186 ip-10-0-135-187 kubenswrapper[2561]: E0716 05:22:00.807168 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2eef8b43bcc782aa8aebdac0d21b4f8f03c3ed47107aa3b1fc02bb1a99f9a07b\": container with ID starting with 2eef8b43bcc782aa8aebdac0d21b4f8f03c3ed47107aa3b1fc02bb1a99f9a07b not found: ID does not exist" containerID="2eef8b43bcc782aa8aebdac0d21b4f8f03c3ed47107aa3b1fc02bb1a99f9a07b" Jul 16 05:22:00.807232 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.807193 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eef8b43bcc782aa8aebdac0d21b4f8f03c3ed47107aa3b1fc02bb1a99f9a07b"} err="failed to get container status \"2eef8b43bcc782aa8aebdac0d21b4f8f03c3ed47107aa3b1fc02bb1a99f9a07b\": rpc error: code = NotFound desc = could not find container \"2eef8b43bcc782aa8aebdac0d21b4f8f03c3ed47107aa3b1fc02bb1a99f9a07b\": container with ID starting with 2eef8b43bcc782aa8aebdac0d21b4f8f03c3ed47107aa3b1fc02bb1a99f9a07b not found: ID does not exist" Jul 16 05:22:00.825453 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.825423 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istiod-ca-cert\") pod \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " Jul 16 05:22:00.825543 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.825493 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istio-podinfo\") pod \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " Jul 16 05:22:00.825543 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.825518 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-workload-socket\") pod \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " Jul 16 05:22:00.825543 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.825535 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-workload-certs\") pod \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " Jul 16 05:22:00.825713 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.825552 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istio-data\") pod \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " Jul 16 05:22:00.825713 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.825605 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-credential-socket\") pod \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " Jul 16 05:22:00.825713 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.825654 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istio-envoy\") pod \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " Jul 16 05:22:00.825848 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.825714 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vkzl\" (UniqueName: \"kubernetes.io/projected/ba7c5dbb-e277-4480-8222-dd5a372a72f5-kube-api-access-4vkzl\") pod \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " Jul 16 05:22:00.825848 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.825737 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istio-token\") pod \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\" (UID: \"ba7c5dbb-e277-4480-8222-dd5a372a72f5\") " Jul 16 05:22:00.825972 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.825946 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ba7c5dbb-e277-4480-8222-dd5a372a72f5" (UID: "ba7c5dbb-e277-4480-8222-dd5a372a72f5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:22:00.826092 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.826068 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ba7c5dbb-e277-4480-8222-dd5a372a72f5" (UID: "ba7c5dbb-e277-4480-8222-dd5a372a72f5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:22:00.826230 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.826211 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ba7c5dbb-e277-4480-8222-dd5a372a72f5" (UID: "ba7c5dbb-e277-4480-8222-dd5a372a72f5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 05:22:00.826230 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.826219 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ba7c5dbb-e277-4480-8222-dd5a372a72f5" (UID: "ba7c5dbb-e277-4480-8222-dd5a372a72f5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:22:00.826358 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.826297 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istiod-ca-cert\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:22:00.826525 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.826510 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-workload-socket\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:22:00.826596 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.826532 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-workload-certs\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:22:00.826596 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.826546 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-credential-socket\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:22:00.826720 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.826651 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istio-data" (OuterVolumeSpecName: "istio-data") pod "ba7c5dbb-e277-4480-8222-dd5a372a72f5" (UID: "ba7c5dbb-e277-4480-8222-dd5a372a72f5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:22:00.827884 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.827860 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ba7c5dbb-e277-4480-8222-dd5a372a72f5" (UID: "ba7c5dbb-e277-4480-8222-dd5a372a72f5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:22:00.828407 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.828389 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba7c5dbb-e277-4480-8222-dd5a372a72f5-kube-api-access-4vkzl" (OuterVolumeSpecName: "kube-api-access-4vkzl") pod "ba7c5dbb-e277-4480-8222-dd5a372a72f5" (UID: "ba7c5dbb-e277-4480-8222-dd5a372a72f5"). InnerVolumeSpecName "kube-api-access-4vkzl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:22:00.828525 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.828505 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istio-token" (OuterVolumeSpecName: "istio-token") pod "ba7c5dbb-e277-4480-8222-dd5a372a72f5" (UID: "ba7c5dbb-e277-4480-8222-dd5a372a72f5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:22:00.828588 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.828503 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ba7c5dbb-e277-4480-8222-dd5a372a72f5" (UID: "ba7c5dbb-e277-4480-8222-dd5a372a72f5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 05:22:00.927954 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.927922 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istio-podinfo\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:22:00.927954 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.927949 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istio-data\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:22:00.927954 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.927959 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istio-envoy\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:22:00.928185 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.927972 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vkzl\" (UniqueName: \"kubernetes.io/projected/ba7c5dbb-e277-4480-8222-dd5a372a72f5-kube-api-access-4vkzl\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:22:00.928185 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:00.927986 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba7c5dbb-e277-4480-8222-dd5a372a72f5-istio-token\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:22:01.110747 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.110714 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j"] Jul 16 05:22:01.114875 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.114851 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-1eae9b50-gw-openshift-default-7c6759678c-lsg8j"] Jul 16 05:22:01.683267 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.683237 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4"] Jul 16 05:22:01.683591 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.683577 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92ca8168-8aa4-4d16-b30c-f65ebb217a9c" containerName="istio-proxy" Jul 16 05:22:01.683632 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.683596 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="92ca8168-8aa4-4d16-b30c-f65ebb217a9c" containerName="istio-proxy" Jul 16 05:22:01.683632 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.683614 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba7c5dbb-e277-4480-8222-dd5a372a72f5" containerName="istio-proxy" Jul 16 05:22:01.683632 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.683622 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba7c5dbb-e277-4480-8222-dd5a372a72f5" containerName="istio-proxy" Jul 16 05:22:01.683766 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.683709 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba7c5dbb-e277-4480-8222-dd5a372a72f5" containerName="istio-proxy" Jul 16 05:22:01.683766 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.683721 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="92ca8168-8aa4-4d16-b30c-f65ebb217a9c" containerName="istio-proxy" Jul 16 05:22:01.683766 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.683728 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="92ca8168-8aa4-4d16-b30c-f65ebb217a9c" containerName="istio-proxy" Jul 16 05:22:01.688084 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.688065 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.690812 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.690791 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-71d327bc-openshift-default-dockercfg-nsfz8\"" Jul 16 05:22:01.700422 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.700400 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4"] Jul 16 05:22:01.836624 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.836587 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f398c603-a6e9-4379-b89f-652a3ca93774-istio-podinfo\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.836624 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.836623 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5srt7\" (UniqueName: \"kubernetes.io/projected/f398c603-a6e9-4379-b89f-652a3ca93774-kube-api-access-5srt7\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.837054 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.836643 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-credential-socket\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.837054 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.836739 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-workload-certs\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.837054 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.836762 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-istio-envoy\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.837054 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.836845 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-istio-data\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.837054 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.836883 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f398c603-a6e9-4379-b89f-652a3ca93774-istio-token\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.837054 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.836899 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f398c603-a6e9-4379-b89f-652a3ca93774-istiod-ca-cert\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.837054 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.836940 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-workload-socket\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.937906 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.937815 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-workload-socket\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.937906 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.937870 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f398c603-a6e9-4379-b89f-652a3ca93774-istio-podinfo\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.937906 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.937889 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5srt7\" (UniqueName: \"kubernetes.io/projected/f398c603-a6e9-4379-b89f-652a3ca93774-kube-api-access-5srt7\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.938116 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.938020 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-credential-socket\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.938116 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.938091 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-workload-certs\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.938191 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.938134 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-istio-envoy\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.938230 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.938216 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-istio-data\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.938273 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.938222 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-workload-socket\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.938273 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.938254 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f398c603-a6e9-4379-b89f-652a3ca93774-istio-token\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.938372 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.938280 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f398c603-a6e9-4379-b89f-652a3ca93774-istiod-ca-cert\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.938372 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.938344 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-credential-socket\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.938472 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.938403 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-workload-certs\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.938527 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.938484 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-istio-data\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.939202 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.939181 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f398c603-a6e9-4379-b89f-652a3ca93774-istiod-ca-cert\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.940470 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.940449 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-istio-envoy\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.940637 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.940617 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f398c603-a6e9-4379-b89f-652a3ca93774-istio-podinfo\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.946190 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.946167 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f398c603-a6e9-4379-b89f-652a3ca93774-istio-token\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.946375 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.946353 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5srt7\" (UniqueName: \"kubernetes.io/projected/f398c603-a6e9-4379-b89f-652a3ca93774-kube-api-access-5srt7\") pod \"e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:01.998217 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:01.998179 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:02.104513 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:02.104478 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba7c5dbb-e277-4480-8222-dd5a372a72f5" path="/var/lib/kubelet/pods/ba7c5dbb-e277-4480-8222-dd5a372a72f5/volumes" Jul 16 05:22:02.145009 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:02.144980 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4"] Jul 16 05:22:02.147526 ip-10-0-135-187 kubenswrapper[2561]: W0716 05:22:02.147496 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf398c603_a6e9_4379_b89f_652a3ca93774.slice/crio-0172ad89280298f09da511dbad352a912fc82a39eb90388be0808f99c0ed5fea WatchSource:0}: Error finding container 0172ad89280298f09da511dbad352a912fc82a39eb90388be0808f99c0ed5fea: Status 404 returned error can't find the container with id 0172ad89280298f09da511dbad352a912fc82a39eb90388be0808f99c0ed5fea Jul 16 05:22:02.149731 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:02.149673 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:22:02.149803 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:02.149778 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:22:02.149847 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:02.149822 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:22:02.802857 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:02.802821 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" event={"ID":"f398c603-a6e9-4379-b89f-652a3ca93774","Type":"ContainerStarted","Data":"6f46b2ad5796197cf8f2742df435c830341a52b65ce1906cfcce05a4e2b41711"} Jul 16 05:22:02.803036 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:02.802877 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" event={"ID":"f398c603-a6e9-4379-b89f-652a3ca93774","Type":"ContainerStarted","Data":"0172ad89280298f09da511dbad352a912fc82a39eb90388be0808f99c0ed5fea"} Jul 16 05:22:02.831232 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:02.831180 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" podStartSLOduration=1.8311638609999998 podStartE2EDuration="1.831163861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:22:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:22:02.829014536 +0000 UTC m=+1913.201137741" watchObservedRunningTime="2026-07-16 05:22:02.831163861 +0000 UTC m=+1913.203287068" Jul 16 05:22:02.999133 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:02.999102 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:04.003224 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:04.003191 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:04.811927 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:04.811893 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:04.812946 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:04.812922 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:13.851865 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:13.851778 2561 generic.go:356] "Generic (PLEG): container finished" podID="f398c603-a6e9-4379-b89f-652a3ca93774" containerID="6f46b2ad5796197cf8f2742df435c830341a52b65ce1906cfcce05a4e2b41711" exitCode=0 Jul 16 05:22:13.851865 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:13.851824 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" event={"ID":"f398c603-a6e9-4379-b89f-652a3ca93774","Type":"ContainerDied","Data":"6f46b2ad5796197cf8f2742df435c830341a52b65ce1906cfcce05a4e2b41711"} Jul 16 05:22:13.852270 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:13.852193 2561 scope.go:117] "RemoveContainer" containerID="6f46b2ad5796197cf8f2742df435c830341a52b65ce1906cfcce05a4e2b41711" Jul 16 05:22:13.853201 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:13.853169 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:22:13.853287 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:13.853270 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:22:13.853369 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:13.853325 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:22:14.858167 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:14.858129 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" event={"ID":"f398c603-a6e9-4379-b89f-652a3ca93774","Type":"ContainerStarted","Data":"97a56c8355dfa740da39301a5f44c55315a58b76bd7d8452bfae56e466dcb6df"} Jul 16 05:22:14.999277 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:14.999238 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:15.000198 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:15.000172 2561 patch_prober.go:28] interesting pod/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body= Jul 16 05:22:15.000310 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:15.000246 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" podUID="f398c603-a6e9-4379-b89f-652a3ca93774" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" Jul 16 05:22:15.999333 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:15.999287 2561 patch_prober.go:28] interesting pod/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body= Jul 16 05:22:15.999750 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:15.999358 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" podUID="f398c603-a6e9-4379-b89f-652a3ca93774" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" Jul 16 05:22:16.999183 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:16.999147 2561 patch_prober.go:28] interesting pod/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body= Jul 16 05:22:16.999372 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:16.999215 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" podUID="f398c603-a6e9-4379-b89f-652a3ca93774" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" Jul 16 05:22:17.999608 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:17.999560 2561 patch_prober.go:28] interesting pod/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body= Jul 16 05:22:18.000041 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:17.999637 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" podUID="f398c603-a6e9-4379-b89f-652a3ca93774" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" Jul 16 05:22:19.002281 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:19.002250 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:19.002652 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:19.002614 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:22:19.003321 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:22:19.003302 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:23:24.357563 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.357525 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4"] Jul 16 05:23:24.358097 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.357743 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" podUID="f398c603-a6e9-4379-b89f-652a3ca93774" containerName="istio-proxy" containerID="cri-o://97a56c8355dfa740da39301a5f44c55315a58b76bd7d8452bfae56e466dcb6df" gracePeriod=30 Jul 16 05:23:24.726374 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.726294 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6"] Jul 16 05:23:24.726815 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.726795 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92ca8168-8aa4-4d16-b30c-f65ebb217a9c" containerName="istio-proxy" Jul 16 05:23:24.726865 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.726819 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="92ca8168-8aa4-4d16-b30c-f65ebb217a9c" containerName="istio-proxy" Jul 16 05:23:24.726865 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.726838 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba7c5dbb-e277-4480-8222-dd5a372a72f5" containerName="istio-proxy" Jul 16 05:23:24.726865 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.726846 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba7c5dbb-e277-4480-8222-dd5a372a72f5" containerName="istio-proxy" Jul 16 05:23:24.727031 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.726951 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba7c5dbb-e277-4480-8222-dd5a372a72f5" containerName="istio-proxy" Jul 16 05:23:24.730213 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.730194 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.732826 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.732804 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-8142960b-openshift-default-dockercfg-plh49\"" Jul 16 05:23:24.742555 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.742530 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6"] Jul 16 05:23:24.782569 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.782538 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-workload-socket\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.782747 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.782580 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-workload-certs\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.782747 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.782608 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f832cfb-d451-4d42-8933-ea86945e77c4-istio-token\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.782747 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.782679 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-istio-data\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.782747 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.782738 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f832cfb-d451-4d42-8933-ea86945e77c4-istio-podinfo\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.782910 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.782761 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f832cfb-d451-4d42-8933-ea86945e77c4-istiod-ca-cert\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.782910 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.782779 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4dm6\" (UniqueName: \"kubernetes.io/projected/5f832cfb-d451-4d42-8933-ea86945e77c4-kube-api-access-k4dm6\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.782910 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.782821 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-credential-socket\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.782910 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.782875 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-istio-envoy\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.883846 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.883808 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-credential-socket\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.883846 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.883847 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-istio-envoy\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.884084 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.883893 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-workload-socket\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.884084 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.883916 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-workload-certs\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.884084 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.883939 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f832cfb-d451-4d42-8933-ea86945e77c4-istio-token\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.884084 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.883982 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-istio-data\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.884084 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.884014 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f832cfb-d451-4d42-8933-ea86945e77c4-istio-podinfo\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.884084 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.884038 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f832cfb-d451-4d42-8933-ea86945e77c4-istiod-ca-cert\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.884084 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.884062 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4dm6\" (UniqueName: \"kubernetes.io/projected/5f832cfb-d451-4d42-8933-ea86945e77c4-kube-api-access-k4dm6\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.884438 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.884255 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-credential-socket\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.884438 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.884425 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-workload-socket\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.884530 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.884474 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-workload-certs\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.884565 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.884528 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-istio-data\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.885103 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.885073 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f832cfb-d451-4d42-8933-ea86945e77c4-istiod-ca-cert\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.886456 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.886435 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-istio-envoy\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.886713 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.886672 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f832cfb-d451-4d42-8933-ea86945e77c4-istio-podinfo\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.892204 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.892179 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f832cfb-d451-4d42-8933-ea86945e77c4-istio-token\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:24.892589 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:24.892568 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4dm6\" (UniqueName: \"kubernetes.io/projected/5f832cfb-d451-4d42-8933-ea86945e77c4-kube-api-access-k4dm6\") pod \"e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:25.044214 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:25.044184 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:25.180223 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:25.180201 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6"] Jul 16 05:23:25.182202 ip-10-0-135-187 kubenswrapper[2561]: W0716 05:23:25.182173 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f832cfb_d451_4d42_8933_ea86945e77c4.slice/crio-6e2a3df10f874f324a7d50d11558021bc0b654f2e63bd41e22b393b04857e406 WatchSource:0}: Error finding container 6e2a3df10f874f324a7d50d11558021bc0b654f2e63bd41e22b393b04857e406: Status 404 returned error can't find the container with id 6e2a3df10f874f324a7d50d11558021bc0b654f2e63bd41e22b393b04857e406 Jul 16 05:23:25.184429 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:25.184396 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:23:25.184535 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:25.184476 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:23:25.184596 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:25.184544 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:23:26.175667 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:26.175629 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" event={"ID":"5f832cfb-d451-4d42-8933-ea86945e77c4","Type":"ContainerStarted","Data":"b21496385208475c96624c0665622d41f05b973ad320084263f3f4915e2a4b5a"} Jul 16 05:23:26.176164 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:26.175697 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" event={"ID":"5f832cfb-d451-4d42-8933-ea86945e77c4","Type":"ContainerStarted","Data":"6e2a3df10f874f324a7d50d11558021bc0b654f2e63bd41e22b393b04857e406"} Jul 16 05:23:26.196484 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:26.196435 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" podStartSLOduration=2.196420916 podStartE2EDuration="2.196420916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:23:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:23:26.194719596 +0000 UTC m=+1996.566842808" watchObservedRunningTime="2026-07-16 05:23:26.196420916 +0000 UTC m=+1996.568544121" Jul 16 05:23:27.044660 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:27.044624 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:27.049418 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:27.049392 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:27.180037 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:27.180005 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:27.180955 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:27.180936 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:29.601449 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.601423 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:23:29.730775 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.730741 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-workload-socket\") pod \"f398c603-a6e9-4379-b89f-652a3ca93774\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " Jul 16 05:23:29.731019 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.730842 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f398c603-a6e9-4379-b89f-652a3ca93774-istio-podinfo\") pod \"f398c603-a6e9-4379-b89f-652a3ca93774\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " Jul 16 05:23:29.731019 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.730870 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f398c603-a6e9-4379-b89f-652a3ca93774-istiod-ca-cert\") pod \"f398c603-a6e9-4379-b89f-652a3ca93774\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " Jul 16 05:23:29.731019 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.730886 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-istio-envoy\") pod \"f398c603-a6e9-4379-b89f-652a3ca93774\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " Jul 16 05:23:29.731019 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.730914 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5srt7\" (UniqueName: \"kubernetes.io/projected/f398c603-a6e9-4379-b89f-652a3ca93774-kube-api-access-5srt7\") pod \"f398c603-a6e9-4379-b89f-652a3ca93774\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " Jul 16 05:23:29.731019 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.730963 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-credential-socket\") pod \"f398c603-a6e9-4379-b89f-652a3ca93774\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " Jul 16 05:23:29.731019 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.730989 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-workload-certs\") pod \"f398c603-a6e9-4379-b89f-652a3ca93774\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " Jul 16 05:23:29.731337 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.731026 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f398c603-a6e9-4379-b89f-652a3ca93774-istio-token\") pod \"f398c603-a6e9-4379-b89f-652a3ca93774\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " Jul 16 05:23:29.731337 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.731056 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-istio-data\") pod \"f398c603-a6e9-4379-b89f-652a3ca93774\" (UID: \"f398c603-a6e9-4379-b89f-652a3ca93774\") " Jul 16 05:23:29.731337 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.731062 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f398c603-a6e9-4379-b89f-652a3ca93774" (UID: "f398c603-a6e9-4379-b89f-652a3ca93774"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:23:29.731337 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.731262 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f398c603-a6e9-4379-b89f-652a3ca93774" (UID: "f398c603-a6e9-4379-b89f-652a3ca93774"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:23:29.731337 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.731318 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f398c603-a6e9-4379-b89f-652a3ca93774" (UID: "f398c603-a6e9-4379-b89f-652a3ca93774"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:23:29.731577 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.731361 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f398c603-a6e9-4379-b89f-652a3ca93774-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f398c603-a6e9-4379-b89f-652a3ca93774" (UID: "f398c603-a6e9-4379-b89f-652a3ca93774"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 05:23:29.731577 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.731412 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-credential-socket\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:23:29.731577 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.731432 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-workload-certs\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:23:29.731577 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.731445 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-workload-socket\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:23:29.732343 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.732314 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-istio-data" (OuterVolumeSpecName: "istio-data") pod "f398c603-a6e9-4379-b89f-652a3ca93774" (UID: "f398c603-a6e9-4379-b89f-652a3ca93774"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:23:29.733280 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.733241 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f398c603-a6e9-4379-b89f-652a3ca93774" (UID: "f398c603-a6e9-4379-b89f-652a3ca93774"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:23:29.733407 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.733309 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f398c603-a6e9-4379-b89f-652a3ca93774-kube-api-access-5srt7" (OuterVolumeSpecName: "kube-api-access-5srt7") pod "f398c603-a6e9-4379-b89f-652a3ca93774" (UID: "f398c603-a6e9-4379-b89f-652a3ca93774"). InnerVolumeSpecName "kube-api-access-5srt7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:23:29.733407 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.733362 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f398c603-a6e9-4379-b89f-652a3ca93774-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f398c603-a6e9-4379-b89f-652a3ca93774" (UID: "f398c603-a6e9-4379-b89f-652a3ca93774"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 05:23:29.733530 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.733515 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f398c603-a6e9-4379-b89f-652a3ca93774-istio-token" (OuterVolumeSpecName: "istio-token") pod "f398c603-a6e9-4379-b89f-652a3ca93774" (UID: "f398c603-a6e9-4379-b89f-652a3ca93774"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:23:29.832931 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.832895 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f398c603-a6e9-4379-b89f-652a3ca93774-istio-podinfo\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:23:29.832931 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.832927 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f398c603-a6e9-4379-b89f-652a3ca93774-istiod-ca-cert\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:23:29.832931 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.832937 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-istio-envoy\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:23:29.833172 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.832945 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5srt7\" (UniqueName: \"kubernetes.io/projected/f398c603-a6e9-4379-b89f-652a3ca93774-kube-api-access-5srt7\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:23:29.833172 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.832956 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f398c603-a6e9-4379-b89f-652a3ca93774-istio-token\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:23:29.833172 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:29.832963 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f398c603-a6e9-4379-b89f-652a3ca93774-istio-data\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:23:30.197040 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:30.197007 2561 generic.go:356] "Generic (PLEG): container finished" podID="f398c603-a6e9-4379-b89f-652a3ca93774" containerID="97a56c8355dfa740da39301a5f44c55315a58b76bd7d8452bfae56e466dcb6df" exitCode=0 Jul 16 05:23:30.197216 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:30.197060 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" Jul 16 05:23:30.197216 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:30.197093 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" event={"ID":"f398c603-a6e9-4379-b89f-652a3ca93774","Type":"ContainerDied","Data":"97a56c8355dfa740da39301a5f44c55315a58b76bd7d8452bfae56e466dcb6df"} Jul 16 05:23:30.197216 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:30.197172 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4" event={"ID":"f398c603-a6e9-4379-b89f-652a3ca93774","Type":"ContainerDied","Data":"0172ad89280298f09da511dbad352a912fc82a39eb90388be0808f99c0ed5fea"} Jul 16 05:23:30.197216 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:30.197201 2561 scope.go:117] "RemoveContainer" containerID="97a56c8355dfa740da39301a5f44c55315a58b76bd7d8452bfae56e466dcb6df" Jul 16 05:23:30.208267 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:30.208247 2561 scope.go:117] "RemoveContainer" containerID="6f46b2ad5796197cf8f2742df435c830341a52b65ce1906cfcce05a4e2b41711" Jul 16 05:23:30.215820 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:30.215797 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4"] Jul 16 05:23:30.216175 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:30.216163 2561 scope.go:117] "RemoveContainer" containerID="97a56c8355dfa740da39301a5f44c55315a58b76bd7d8452bfae56e466dcb6df" Jul 16 05:23:30.216438 ip-10-0-135-187 kubenswrapper[2561]: E0716 05:23:30.216412 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97a56c8355dfa740da39301a5f44c55315a58b76bd7d8452bfae56e466dcb6df\": container with ID starting with 97a56c8355dfa740da39301a5f44c55315a58b76bd7d8452bfae56e466dcb6df not found: ID does not exist" containerID="97a56c8355dfa740da39301a5f44c55315a58b76bd7d8452bfae56e466dcb6df" Jul 16 05:23:30.216504 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:30.216448 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97a56c8355dfa740da39301a5f44c55315a58b76bd7d8452bfae56e466dcb6df"} err="failed to get container status \"97a56c8355dfa740da39301a5f44c55315a58b76bd7d8452bfae56e466dcb6df\": rpc error: code = NotFound desc = could not find container \"97a56c8355dfa740da39301a5f44c55315a58b76bd7d8452bfae56e466dcb6df\": container with ID starting with 97a56c8355dfa740da39301a5f44c55315a58b76bd7d8452bfae56e466dcb6df not found: ID does not exist" Jul 16 05:23:30.216504 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:30.216467 2561 scope.go:117] "RemoveContainer" containerID="6f46b2ad5796197cf8f2742df435c830341a52b65ce1906cfcce05a4e2b41711" Jul 16 05:23:30.216725 ip-10-0-135-187 kubenswrapper[2561]: E0716 05:23:30.216707 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f46b2ad5796197cf8f2742df435c830341a52b65ce1906cfcce05a4e2b41711\": container with ID starting with 6f46b2ad5796197cf8f2742df435c830341a52b65ce1906cfcce05a4e2b41711 not found: ID does not exist" containerID="6f46b2ad5796197cf8f2742df435c830341a52b65ce1906cfcce05a4e2b41711" Jul 16 05:23:30.216773 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:30.216731 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f46b2ad5796197cf8f2742df435c830341a52b65ce1906cfcce05a4e2b41711"} err="failed to get container status \"6f46b2ad5796197cf8f2742df435c830341a52b65ce1906cfcce05a4e2b41711\": rpc error: code = NotFound desc = could not find container \"6f46b2ad5796197cf8f2742df435c830341a52b65ce1906cfcce05a4e2b41711\": container with ID starting with 6f46b2ad5796197cf8f2742df435c830341a52b65ce1906cfcce05a4e2b41711 not found: ID does not exist" Jul 16 05:23:30.219381 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:30.219362 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-71d327bc-openshift-default-5844d7bf57-x4pl4"] Jul 16 05:23:32.101586 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:32.101542 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f398c603-a6e9-4379-b89f-652a3ca93774" path="/var/lib/kubelet/pods/f398c603-a6e9-4379-b89f-652a3ca93774/volumes" Jul 16 05:23:42.181043 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:42.181006 2561 patch_prober.go:28] interesting pod/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 16 05:23:42.181457 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:42.181074 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" podUID="5f832cfb-d451-4d42-8933-ea86945e77c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 16 05:23:42.254772 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:42.254734 2561 generic.go:356] "Generic (PLEG): container finished" podID="5f832cfb-d451-4d42-8933-ea86945e77c4" containerID="b21496385208475c96624c0665622d41f05b973ad320084263f3f4915e2a4b5a" exitCode=0 Jul 16 05:23:42.254772 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:42.254773 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" event={"ID":"5f832cfb-d451-4d42-8933-ea86945e77c4","Type":"ContainerDied","Data":"b21496385208475c96624c0665622d41f05b973ad320084263f3f4915e2a4b5a"} Jul 16 05:23:42.255148 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:42.255134 2561 scope.go:117] "RemoveContainer" containerID="b21496385208475c96624c0665622d41f05b973ad320084263f3f4915e2a4b5a" Jul 16 05:23:42.256119 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:42.256095 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:23:42.256173 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:42.256160 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:23:42.256207 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:42.256192 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:23:43.261193 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:43.261162 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" event={"ID":"5f832cfb-d451-4d42-8933-ea86945e77c4","Type":"ContainerStarted","Data":"3846e13c14b52d7bc5fa43d8e144d2ecc6e1c08bab4df05eaec178d3c75067da"} Jul 16 05:23:44.044552 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:44.044515 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:44.044756 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:44.044667 2561 patch_prober.go:28] interesting pod/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 16 05:23:44.044756 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:44.044736 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" podUID="5f832cfb-d451-4d42-8933-ea86945e77c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 16 05:23:45.044573 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:45.044537 2561 patch_prober.go:28] interesting pod/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 16 05:23:45.044998 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:45.044591 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" podUID="5f832cfb-d451-4d42-8933-ea86945e77c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 16 05:23:46.045508 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:46.045466 2561 patch_prober.go:28] interesting pod/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 16 05:23:46.045922 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:46.045529 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" podUID="5f832cfb-d451-4d42-8933-ea86945e77c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 16 05:23:47.078398 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:47.078361 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:47.078797 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:47.078612 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:23:47.079277 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:23:47.079261 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:24:45.685073 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:45.684982 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6"] Jul 16 05:24:45.685671 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:45.685357 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" podUID="5f832cfb-d451-4d42-8933-ea86945e77c4" containerName="istio-proxy" containerID="cri-o://3846e13c14b52d7bc5fa43d8e144d2ecc6e1c08bab4df05eaec178d3c75067da" gracePeriod=30 Jul 16 05:24:47.080936 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:47.080891 2561 patch_prober.go:28] interesting pod/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 05:24:47.081349 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:47.080980 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" podUID="5f832cfb-d451-4d42-8933-ea86945e77c4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 05:24:50.933905 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:50.933879 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:24:51.073066 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.073030 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4dm6\" (UniqueName: \"kubernetes.io/projected/5f832cfb-d451-4d42-8933-ea86945e77c4-kube-api-access-k4dm6\") pod \"5f832cfb-d451-4d42-8933-ea86945e77c4\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " Jul 16 05:24:51.073066 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.073069 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f832cfb-d451-4d42-8933-ea86945e77c4-istio-token\") pod \"5f832cfb-d451-4d42-8933-ea86945e77c4\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " Jul 16 05:24:51.073290 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.073095 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-credential-socket\") pod \"5f832cfb-d451-4d42-8933-ea86945e77c4\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " Jul 16 05:24:51.073290 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.073132 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-istio-data\") pod \"5f832cfb-d451-4d42-8933-ea86945e77c4\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " Jul 16 05:24:51.073290 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.073151 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-istio-envoy\") pod \"5f832cfb-d451-4d42-8933-ea86945e77c4\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " Jul 16 05:24:51.073290 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.073196 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-workload-socket\") pod \"5f832cfb-d451-4d42-8933-ea86945e77c4\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " Jul 16 05:24:51.073290 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.073215 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f832cfb-d451-4d42-8933-ea86945e77c4-istio-podinfo\") pod \"5f832cfb-d451-4d42-8933-ea86945e77c4\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " Jul 16 05:24:51.073290 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.073267 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-workload-certs\") pod \"5f832cfb-d451-4d42-8933-ea86945e77c4\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " Jul 16 05:24:51.073290 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.073292 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f832cfb-d451-4d42-8933-ea86945e77c4-istiod-ca-cert\") pod \"5f832cfb-d451-4d42-8933-ea86945e77c4\" (UID: \"5f832cfb-d451-4d42-8933-ea86945e77c4\") " Jul 16 05:24:51.073629 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.073422 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5f832cfb-d451-4d42-8933-ea86945e77c4" (UID: "5f832cfb-d451-4d42-8933-ea86945e77c4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:24:51.073629 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.073578 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-credential-socket\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:24:51.073756 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.073639 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5f832cfb-d451-4d42-8933-ea86945e77c4" (UID: "5f832cfb-d451-4d42-8933-ea86945e77c4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:24:51.073842 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.073811 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5f832cfb-d451-4d42-8933-ea86945e77c4" (UID: "5f832cfb-d451-4d42-8933-ea86945e77c4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:24:51.074135 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.073891 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f832cfb-d451-4d42-8933-ea86945e77c4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5f832cfb-d451-4d42-8933-ea86945e77c4" (UID: "5f832cfb-d451-4d42-8933-ea86945e77c4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 05:24:51.074224 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.074195 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-istio-data" (OuterVolumeSpecName: "istio-data") pod "5f832cfb-d451-4d42-8933-ea86945e77c4" (UID: "5f832cfb-d451-4d42-8933-ea86945e77c4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:24:51.075571 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.075543 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f832cfb-d451-4d42-8933-ea86945e77c4-istio-token" (OuterVolumeSpecName: "istio-token") pod "5f832cfb-d451-4d42-8933-ea86945e77c4" (UID: "5f832cfb-d451-4d42-8933-ea86945e77c4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:24:51.075571 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.075558 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5f832cfb-d451-4d42-8933-ea86945e77c4" (UID: "5f832cfb-d451-4d42-8933-ea86945e77c4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:24:51.075727 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.075543 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f832cfb-d451-4d42-8933-ea86945e77c4-kube-api-access-k4dm6" (OuterVolumeSpecName: "kube-api-access-k4dm6") pod "5f832cfb-d451-4d42-8933-ea86945e77c4" (UID: "5f832cfb-d451-4d42-8933-ea86945e77c4"). InnerVolumeSpecName "kube-api-access-k4dm6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:24:51.075798 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.075732 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5f832cfb-d451-4d42-8933-ea86945e77c4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5f832cfb-d451-4d42-8933-ea86945e77c4" (UID: "5f832cfb-d451-4d42-8933-ea86945e77c4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 05:24:51.174898 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.174854 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4dm6\" (UniqueName: \"kubernetes.io/projected/5f832cfb-d451-4d42-8933-ea86945e77c4-kube-api-access-k4dm6\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:24:51.174898 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.174892 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f832cfb-d451-4d42-8933-ea86945e77c4-istio-token\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:24:51.174898 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.174903 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-istio-data\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:24:51.174898 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.174911 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-istio-envoy\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:24:51.175218 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.174919 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-workload-socket\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:24:51.175218 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.174927 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f832cfb-d451-4d42-8933-ea86945e77c4-istio-podinfo\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:24:51.175218 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.174936 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f832cfb-d451-4d42-8933-ea86945e77c4-workload-certs\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:24:51.175218 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.174945 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f832cfb-d451-4d42-8933-ea86945e77c4-istiod-ca-cert\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:24:51.563107 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.563074 2561 generic.go:356] "Generic (PLEG): container finished" podID="5f832cfb-d451-4d42-8933-ea86945e77c4" containerID="3846e13c14b52d7bc5fa43d8e144d2ecc6e1c08bab4df05eaec178d3c75067da" exitCode=0 Jul 16 05:24:51.563282 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.563144 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" Jul 16 05:24:51.563282 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.563152 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" event={"ID":"5f832cfb-d451-4d42-8933-ea86945e77c4","Type":"ContainerDied","Data":"3846e13c14b52d7bc5fa43d8e144d2ecc6e1c08bab4df05eaec178d3c75067da"} Jul 16 05:24:51.563282 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.563220 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6" event={"ID":"5f832cfb-d451-4d42-8933-ea86945e77c4","Type":"ContainerDied","Data":"6e2a3df10f874f324a7d50d11558021bc0b654f2e63bd41e22b393b04857e406"} Jul 16 05:24:51.563282 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.563242 2561 scope.go:117] "RemoveContainer" containerID="3846e13c14b52d7bc5fa43d8e144d2ecc6e1c08bab4df05eaec178d3c75067da" Jul 16 05:24:51.572271 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.572250 2561 scope.go:117] "RemoveContainer" containerID="b21496385208475c96624c0665622d41f05b973ad320084263f3f4915e2a4b5a" Jul 16 05:24:51.579935 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.579916 2561 scope.go:117] "RemoveContainer" containerID="3846e13c14b52d7bc5fa43d8e144d2ecc6e1c08bab4df05eaec178d3c75067da" Jul 16 05:24:51.580191 ip-10-0-135-187 kubenswrapper[2561]: E0716 05:24:51.580171 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3846e13c14b52d7bc5fa43d8e144d2ecc6e1c08bab4df05eaec178d3c75067da\": container with ID starting with 3846e13c14b52d7bc5fa43d8e144d2ecc6e1c08bab4df05eaec178d3c75067da not found: ID does not exist" containerID="3846e13c14b52d7bc5fa43d8e144d2ecc6e1c08bab4df05eaec178d3c75067da" Jul 16 05:24:51.580240 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.580201 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3846e13c14b52d7bc5fa43d8e144d2ecc6e1c08bab4df05eaec178d3c75067da"} err="failed to get container status \"3846e13c14b52d7bc5fa43d8e144d2ecc6e1c08bab4df05eaec178d3c75067da\": rpc error: code = NotFound desc = could not find container \"3846e13c14b52d7bc5fa43d8e144d2ecc6e1c08bab4df05eaec178d3c75067da\": container with ID starting with 3846e13c14b52d7bc5fa43d8e144d2ecc6e1c08bab4df05eaec178d3c75067da not found: ID does not exist" Jul 16 05:24:51.580240 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.580219 2561 scope.go:117] "RemoveContainer" containerID="b21496385208475c96624c0665622d41f05b973ad320084263f3f4915e2a4b5a" Jul 16 05:24:51.580471 ip-10-0-135-187 kubenswrapper[2561]: E0716 05:24:51.580453 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b21496385208475c96624c0665622d41f05b973ad320084263f3f4915e2a4b5a\": container with ID starting with b21496385208475c96624c0665622d41f05b973ad320084263f3f4915e2a4b5a not found: ID does not exist" containerID="b21496385208475c96624c0665622d41f05b973ad320084263f3f4915e2a4b5a" Jul 16 05:24:51.580513 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.580478 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b21496385208475c96624c0665622d41f05b973ad320084263f3f4915e2a4b5a"} err="failed to get container status \"b21496385208475c96624c0665622d41f05b973ad320084263f3f4915e2a4b5a\": rpc error: code = NotFound desc = could not find container \"b21496385208475c96624c0665622d41f05b973ad320084263f3f4915e2a4b5a\": container with ID starting with b21496385208475c96624c0665622d41f05b973ad320084263f3f4915e2a4b5a not found: ID does not exist" Jul 16 05:24:51.583733 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.583710 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6"] Jul 16 05:24:51.589744 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:51.589725 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-8142960b-openshift-default-bcd9f695-x5cp6"] Jul 16 05:24:52.102005 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:24:52.101970 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f832cfb-d451-4d42-8933-ea86945e77c4" path="/var/lib/kubelet/pods/5f832cfb-d451-4d42-8933-ea86945e77c4/volumes" Jul 16 05:26:21.260522 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.260485 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x"] Jul 16 05:26:21.261130 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.260963 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f398c603-a6e9-4379-b89f-652a3ca93774" containerName="istio-proxy" Jul 16 05:26:21.261130 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.260979 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f398c603-a6e9-4379-b89f-652a3ca93774" containerName="istio-proxy" Jul 16 05:26:21.261130 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.260999 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f832cfb-d451-4d42-8933-ea86945e77c4" containerName="istio-proxy" Jul 16 05:26:21.261130 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.261008 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f832cfb-d451-4d42-8933-ea86945e77c4" containerName="istio-proxy" Jul 16 05:26:21.261130 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.261019 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f398c603-a6e9-4379-b89f-652a3ca93774" containerName="istio-proxy" Jul 16 05:26:21.261130 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.261028 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f398c603-a6e9-4379-b89f-652a3ca93774" containerName="istio-proxy" Jul 16 05:26:21.261130 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.261048 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f832cfb-d451-4d42-8933-ea86945e77c4" containerName="istio-proxy" Jul 16 05:26:21.261130 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.261057 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f832cfb-d451-4d42-8933-ea86945e77c4" containerName="istio-proxy" Jul 16 05:26:21.261521 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.261142 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f832cfb-d451-4d42-8933-ea86945e77c4" containerName="istio-proxy" Jul 16 05:26:21.261521 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.261154 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f398c603-a6e9-4379-b89f-652a3ca93774" containerName="istio-proxy" Jul 16 05:26:21.261521 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.261163 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f398c603-a6e9-4379-b89f-652a3ca93774" containerName="istio-proxy" Jul 16 05:26:21.264273 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.264251 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.266785 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.266766 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-44053aaf-openshift-default-dockercfg-p6rpw\"" Jul 16 05:26:21.278720 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.278669 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x"] Jul 16 05:26:21.361423 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.361386 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42rwv\" (UniqueName: \"kubernetes.io/projected/7dbb07fa-ca64-495c-aa34-f978d3f0b175-kube-api-access-42rwv\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.361597 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.361448 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istiod-ca-cert\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.361597 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.361501 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-credential-socket\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.361597 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.361583 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istio-token\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.361742 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.361632 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istio-envoy\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.361742 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.361675 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istio-data\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.361742 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.361737 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-workload-certs\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.361853 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.361769 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-workload-socket\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.361853 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.361801 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istio-podinfo\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.463253 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.463215 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istio-data\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.463429 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.463258 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-workload-certs\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.463429 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.463284 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-workload-socket\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.463429 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.463318 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istio-podinfo\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.463429 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.463351 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42rwv\" (UniqueName: \"kubernetes.io/projected/7dbb07fa-ca64-495c-aa34-f978d3f0b175-kube-api-access-42rwv\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.463429 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.463385 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istiod-ca-cert\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.463429 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.463409 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-credential-socket\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.463776 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.463467 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istio-token\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.463776 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.463510 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istio-envoy\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.463776 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.463667 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istio-data\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.463929 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.463911 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-workload-certs\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.463995 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.463964 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-credential-socket\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.464545 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.464521 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istiod-ca-cert\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.464660 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.464642 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-workload-socket\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.465800 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.465780 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istio-envoy\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.466045 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.466029 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istio-podinfo\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.470942 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.470921 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istio-token\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.471171 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.471153 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42rwv\" (UniqueName: \"kubernetes.io/projected/7dbb07fa-ca64-495c-aa34-f978d3f0b175-kube-api-access-42rwv\") pod \"e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.574947 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.574907 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:21.727008 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.726974 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x"] Jul 16 05:26:21.728231 ip-10-0-135-187 kubenswrapper[2561]: W0716 05:26:21.728207 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dbb07fa_ca64_495c_aa34_f978d3f0b175.slice/crio-4d82f5ce1fbc508e4a40f97acdb40616bedc3050b5e919916f1409991d6bb4e6 WatchSource:0}: Error finding container 4d82f5ce1fbc508e4a40f97acdb40616bedc3050b5e919916f1409991d6bb4e6: Status 404 returned error can't find the container with id 4d82f5ce1fbc508e4a40f97acdb40616bedc3050b5e919916f1409991d6bb4e6 Jul 16 05:26:21.730810 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.730781 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:26:21.730871 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.730851 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:26:21.730906 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.730889 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:26:21.949522 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.949423 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" event={"ID":"7dbb07fa-ca64-495c-aa34-f978d3f0b175","Type":"ContainerStarted","Data":"100d3b70a46860e20ad9394bd3af505a3d72cd3b3a8559566c00e760c8790b03"} Jul 16 05:26:21.949522 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.949486 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" event={"ID":"7dbb07fa-ca64-495c-aa34-f978d3f0b175","Type":"ContainerStarted","Data":"4d82f5ce1fbc508e4a40f97acdb40616bedc3050b5e919916f1409991d6bb4e6"} Jul 16 05:26:21.971561 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:21.971496 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" podStartSLOduration=0.971475444 podStartE2EDuration="971.475444ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:26:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:26:21.968521849 +0000 UTC m=+2172.340645079" watchObservedRunningTime="2026-07-16 05:26:21.971475444 +0000 UTC m=+2172.343598650" Jul 16 05:26:22.575990 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:22.575950 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:22.580632 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:22.580606 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:22.953805 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:22.953716 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:22.954789 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:22.954767 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:37.955672 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:37.955642 2561 patch_prober.go:28] interesting pod/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 16 05:26:37.956030 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:37.955725 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" podUID="7dbb07fa-ca64-495c-aa34-f978d3f0b175" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 16 05:26:38.024938 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:38.024906 2561 generic.go:356] "Generic (PLEG): container finished" podID="7dbb07fa-ca64-495c-aa34-f978d3f0b175" containerID="100d3b70a46860e20ad9394bd3af505a3d72cd3b3a8559566c00e760c8790b03" exitCode=0 Jul 16 05:26:38.025142 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:38.024981 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" event={"ID":"7dbb07fa-ca64-495c-aa34-f978d3f0b175","Type":"ContainerDied","Data":"100d3b70a46860e20ad9394bd3af505a3d72cd3b3a8559566c00e760c8790b03"} Jul 16 05:26:38.025461 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:38.025443 2561 scope.go:117] "RemoveContainer" containerID="100d3b70a46860e20ad9394bd3af505a3d72cd3b3a8559566c00e760c8790b03" Jul 16 05:26:38.026586 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:38.026560 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:26:38.026644 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:38.026625 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:26:38.026701 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:38.026656 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:26:39.032595 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:39.032563 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" event={"ID":"7dbb07fa-ca64-495c-aa34-f978d3f0b175","Type":"ContainerStarted","Data":"6571480ade247d77c8e36453b108bc9d4337d0a093e295bdae731882dee5adb4"} Jul 16 05:26:39.575888 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:39.575793 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:39.575888 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:39.575847 2561 patch_prober.go:28] interesting pod/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 16 05:26:39.576162 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:39.575922 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" podUID="7dbb07fa-ca64-495c-aa34-f978d3f0b175" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 16 05:26:40.578152 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:40.578109 2561 patch_prober.go:28] interesting pod/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 16 05:26:40.578640 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:40.578181 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" podUID="7dbb07fa-ca64-495c-aa34-f978d3f0b175" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 16 05:26:41.576422 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:41.575964 2561 patch_prober.go:28] interesting pod/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 16 05:26:41.576422 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:41.576033 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" podUID="7dbb07fa-ca64-495c-aa34-f978d3f0b175" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 16 05:26:42.576166 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:42.576123 2561 patch_prober.go:28] interesting pod/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 16 05:26:42.576835 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:42.576799 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" podUID="7dbb07fa-ca64-495c-aa34-f978d3f0b175" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 16 05:26:43.579100 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:43.579067 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:43.579492 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:43.579473 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:26:43.580249 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:26:43.580227 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:27:02.055431 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:02.055398 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x"] Jul 16 05:27:02.055917 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:02.055633 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" podUID="7dbb07fa-ca64-495c-aa34-f978d3f0b175" containerName="istio-proxy" containerID="cri-o://6571480ade247d77c8e36453b108bc9d4337d0a093e295bdae731882dee5adb4" gracePeriod=30 Jul 16 05:27:07.304382 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.304352 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:27:07.368791 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.368709 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-workload-socket\") pod \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " Jul 16 05:27:07.368791 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.368755 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-credential-socket\") pod \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " Jul 16 05:27:07.368791 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.368786 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istio-data\") pod \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " Jul 16 05:27:07.369053 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.368831 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istio-envoy\") pod \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " Jul 16 05:27:07.369053 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.368863 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-workload-certs\") pod \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " Jul 16 05:27:07.369053 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.368908 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istiod-ca-cert\") pod \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " Jul 16 05:27:07.369053 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.368937 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42rwv\" (UniqueName: \"kubernetes.io/projected/7dbb07fa-ca64-495c-aa34-f978d3f0b175-kube-api-access-42rwv\") pod \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " Jul 16 05:27:07.369053 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.368964 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istio-token\") pod \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " Jul 16 05:27:07.369053 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.368989 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istio-podinfo\") pod \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\" (UID: \"7dbb07fa-ca64-495c-aa34-f978d3f0b175\") " Jul 16 05:27:07.369053 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.369014 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7dbb07fa-ca64-495c-aa34-f978d3f0b175" (UID: "7dbb07fa-ca64-495c-aa34-f978d3f0b175"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:27:07.369408 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.369160 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7dbb07fa-ca64-495c-aa34-f978d3f0b175" (UID: "7dbb07fa-ca64-495c-aa34-f978d3f0b175"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:27:07.369408 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.369220 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7dbb07fa-ca64-495c-aa34-f978d3f0b175" (UID: "7dbb07fa-ca64-495c-aa34-f978d3f0b175"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:27:07.369408 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.369272 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-workload-socket\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:27:07.369408 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.369283 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7dbb07fa-ca64-495c-aa34-f978d3f0b175" (UID: "7dbb07fa-ca64-495c-aa34-f978d3f0b175"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 05:27:07.369408 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.369293 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-credential-socket\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:27:07.370062 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.370038 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istio-data" (OuterVolumeSpecName: "istio-data") pod "7dbb07fa-ca64-495c-aa34-f978d3f0b175" (UID: "7dbb07fa-ca64-495c-aa34-f978d3f0b175"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:27:07.371174 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.371147 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7dbb07fa-ca64-495c-aa34-f978d3f0b175" (UID: "7dbb07fa-ca64-495c-aa34-f978d3f0b175"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:27:07.371396 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.371260 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dbb07fa-ca64-495c-aa34-f978d3f0b175-kube-api-access-42rwv" (OuterVolumeSpecName: "kube-api-access-42rwv") pod "7dbb07fa-ca64-495c-aa34-f978d3f0b175" (UID: "7dbb07fa-ca64-495c-aa34-f978d3f0b175"). InnerVolumeSpecName "kube-api-access-42rwv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:27:07.371714 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.371673 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istio-token" (OuterVolumeSpecName: "istio-token") pod "7dbb07fa-ca64-495c-aa34-f978d3f0b175" (UID: "7dbb07fa-ca64-495c-aa34-f978d3f0b175"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:27:07.371714 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.371671 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7dbb07fa-ca64-495c-aa34-f978d3f0b175" (UID: "7dbb07fa-ca64-495c-aa34-f978d3f0b175"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 05:27:07.470589 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.470552 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istio-token\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:27:07.470589 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.470581 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istio-podinfo\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:27:07.470589 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.470591 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istio-data\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:27:07.470589 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.470600 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istio-envoy\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:27:07.470885 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.470607 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dbb07fa-ca64-495c-aa34-f978d3f0b175-workload-certs\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:27:07.470885 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.470615 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dbb07fa-ca64-495c-aa34-f978d3f0b175-istiod-ca-cert\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:27:07.470885 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:07.470622 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42rwv\" (UniqueName: \"kubernetes.io/projected/7dbb07fa-ca64-495c-aa34-f978d3f0b175-kube-api-access-42rwv\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:27:08.167878 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:08.167845 2561 generic.go:356] "Generic (PLEG): container finished" podID="7dbb07fa-ca64-495c-aa34-f978d3f0b175" containerID="6571480ade247d77c8e36453b108bc9d4337d0a093e295bdae731882dee5adb4" exitCode=0 Jul 16 05:27:08.168046 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:08.167915 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" Jul 16 05:27:08.168046 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:08.167936 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" event={"ID":"7dbb07fa-ca64-495c-aa34-f978d3f0b175","Type":"ContainerDied","Data":"6571480ade247d77c8e36453b108bc9d4337d0a093e295bdae731882dee5adb4"} Jul 16 05:27:08.168046 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:08.167993 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x" event={"ID":"7dbb07fa-ca64-495c-aa34-f978d3f0b175","Type":"ContainerDied","Data":"4d82f5ce1fbc508e4a40f97acdb40616bedc3050b5e919916f1409991d6bb4e6"} Jul 16 05:27:08.168046 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:08.168018 2561 scope.go:117] "RemoveContainer" containerID="6571480ade247d77c8e36453b108bc9d4337d0a093e295bdae731882dee5adb4" Jul 16 05:27:08.176416 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:08.176401 2561 scope.go:117] "RemoveContainer" containerID="100d3b70a46860e20ad9394bd3af505a3d72cd3b3a8559566c00e760c8790b03" Jul 16 05:27:08.183903 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:08.183880 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x"] Jul 16 05:27:08.184640 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:08.184618 2561 scope.go:117] "RemoveContainer" containerID="6571480ade247d77c8e36453b108bc9d4337d0a093e295bdae731882dee5adb4" Jul 16 05:27:08.184975 ip-10-0-135-187 kubenswrapper[2561]: E0716 05:27:08.184955 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6571480ade247d77c8e36453b108bc9d4337d0a093e295bdae731882dee5adb4\": container with ID starting with 6571480ade247d77c8e36453b108bc9d4337d0a093e295bdae731882dee5adb4 not found: ID does not exist" containerID="6571480ade247d77c8e36453b108bc9d4337d0a093e295bdae731882dee5adb4" Jul 16 05:27:08.185146 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:08.184984 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6571480ade247d77c8e36453b108bc9d4337d0a093e295bdae731882dee5adb4"} err="failed to get container status \"6571480ade247d77c8e36453b108bc9d4337d0a093e295bdae731882dee5adb4\": rpc error: code = NotFound desc = could not find container \"6571480ade247d77c8e36453b108bc9d4337d0a093e295bdae731882dee5adb4\": container with ID starting with 6571480ade247d77c8e36453b108bc9d4337d0a093e295bdae731882dee5adb4 not found: ID does not exist" Jul 16 05:27:08.185146 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:08.185008 2561 scope.go:117] "RemoveContainer" containerID="100d3b70a46860e20ad9394bd3af505a3d72cd3b3a8559566c00e760c8790b03" Jul 16 05:27:08.185432 ip-10-0-135-187 kubenswrapper[2561]: E0716 05:27:08.185408 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"100d3b70a46860e20ad9394bd3af505a3d72cd3b3a8559566c00e760c8790b03\": container with ID starting with 100d3b70a46860e20ad9394bd3af505a3d72cd3b3a8559566c00e760c8790b03 not found: ID does not exist" containerID="100d3b70a46860e20ad9394bd3af505a3d72cd3b3a8559566c00e760c8790b03" Jul 16 05:27:08.185526 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:08.185442 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"100d3b70a46860e20ad9394bd3af505a3d72cd3b3a8559566c00e760c8790b03"} err="failed to get container status \"100d3b70a46860e20ad9394bd3af505a3d72cd3b3a8559566c00e760c8790b03\": rpc error: code = NotFound desc = could not find container \"100d3b70a46860e20ad9394bd3af505a3d72cd3b3a8559566c00e760c8790b03\": container with ID starting with 100d3b70a46860e20ad9394bd3af505a3d72cd3b3a8559566c00e760c8790b03 not found: ID does not exist" Jul 16 05:27:08.187309 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:08.187287 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-44053aaf-openshift-default-54d84579f5-sbw5x"] Jul 16 05:27:10.101303 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:10.101268 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7dbb07fa-ca64-495c-aa34-f978d3f0b175" path="/var/lib/kubelet/pods/7dbb07fa-ca64-495c-aa34-f978d3f0b175/volumes" Jul 16 05:27:21.663460 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.663424 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w"] Jul 16 05:27:21.663856 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.663790 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dbb07fa-ca64-495c-aa34-f978d3f0b175" containerName="istio-proxy" Jul 16 05:27:21.663856 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.663806 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dbb07fa-ca64-495c-aa34-f978d3f0b175" containerName="istio-proxy" Jul 16 05:27:21.663927 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.663870 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dbb07fa-ca64-495c-aa34-f978d3f0b175" containerName="istio-proxy" Jul 16 05:27:21.663927 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.663879 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dbb07fa-ca64-495c-aa34-f978d3f0b175" containerName="istio-proxy" Jul 16 05:27:21.663927 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.663886 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f832cfb-d451-4d42-8933-ea86945e77c4" containerName="istio-proxy" Jul 16 05:27:21.666815 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.666794 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.669544 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.669525 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-18c409ba-openshift-default-dockercfg-png8g\"" Jul 16 05:27:21.682841 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.682815 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w"] Jul 16 05:27:21.798522 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.798487 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab9072b8-695b-4832-965c-7bde95599464-istio-token\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.798755 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.798531 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab9072b8-695b-4832-965c-7bde95599464-istio-podinfo\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.798755 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.798555 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx6zg\" (UniqueName: \"kubernetes.io/projected/ab9072b8-695b-4832-965c-7bde95599464-kube-api-access-cx6zg\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.798755 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.798625 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-credential-socket\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.798755 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.798734 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-workload-certs\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.798972 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.798775 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab9072b8-695b-4832-965c-7bde95599464-istiod-ca-cert\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.798972 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.798844 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-workload-socket\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.798972 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.798866 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-istio-envoy\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.798972 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.798933 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-istio-data\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.899769 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.899728 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx6zg\" (UniqueName: \"kubernetes.io/projected/ab9072b8-695b-4832-965c-7bde95599464-kube-api-access-cx6zg\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.899769 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.899771 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-credential-socket\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.900036 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.899799 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-workload-certs\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.900036 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.899828 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab9072b8-695b-4832-965c-7bde95599464-istiod-ca-cert\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.900036 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.899869 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-workload-socket\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.900036 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.899893 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-istio-envoy\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.900036 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.899949 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-istio-data\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.900036 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.900031 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab9072b8-695b-4832-965c-7bde95599464-istio-token\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.900338 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.900066 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab9072b8-695b-4832-965c-7bde95599464-istio-podinfo\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.900338 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.900223 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-credential-socket\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.900338 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.900280 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-workload-certs\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.900498 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.900405 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-workload-socket\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.900498 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.900420 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-istio-data\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.900627 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.900605 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab9072b8-695b-4832-965c-7bde95599464-istiod-ca-cert\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.902303 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.902281 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-istio-envoy\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.902700 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.902668 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab9072b8-695b-4832-965c-7bde95599464-istio-podinfo\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.909886 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.909867 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx6zg\" (UniqueName: \"kubernetes.io/projected/ab9072b8-695b-4832-965c-7bde95599464-kube-api-access-cx6zg\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.910039 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.910022 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab9072b8-695b-4832-965c-7bde95599464-istio-token\") pod \"e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:21.977163 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:21.977078 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:22.115293 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:22.115269 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w"] Jul 16 05:27:22.117245 ip-10-0-135-187 kubenswrapper[2561]: W0716 05:27:22.117207 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab9072b8_695b_4832_965c_7bde95599464.slice/crio-3b9f31c3465745fa55edce75c20d1242100934093fffccc46059c37a33700121 WatchSource:0}: Error finding container 3b9f31c3465745fa55edce75c20d1242100934093fffccc46059c37a33700121: Status 404 returned error can't find the container with id 3b9f31c3465745fa55edce75c20d1242100934093fffccc46059c37a33700121 Jul 16 05:27:22.119569 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:22.119540 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:27:22.119662 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:22.119618 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:27:22.119752 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:22.119662 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:27:22.230521 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:22.230424 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" event={"ID":"ab9072b8-695b-4832-965c-7bde95599464","Type":"ContainerStarted","Data":"3143f8c04e7c290bde7631d0b3aa4e2ad3e318adbfcb94ed036ad71f1866383e"} Jul 16 05:27:22.230521 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:22.230502 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" event={"ID":"ab9072b8-695b-4832-965c-7bde95599464","Type":"ContainerStarted","Data":"3b9f31c3465745fa55edce75c20d1242100934093fffccc46059c37a33700121"} Jul 16 05:27:22.253414 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:22.253353 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" podStartSLOduration=1.253331979 podStartE2EDuration="1.253331979s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:27:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:27:22.250272007 +0000 UTC m=+2232.622395251" watchObservedRunningTime="2026-07-16 05:27:22.253331979 +0000 UTC m=+2232.625455186" Jul 16 05:27:22.977316 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:22.977280 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:23.982427 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:23.982397 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:24.240775 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:24.240674 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:24.241794 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:24.241777 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:37.323261 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:37.323225 2561 generic.go:356] "Generic (PLEG): container finished" podID="ab9072b8-695b-4832-965c-7bde95599464" containerID="3143f8c04e7c290bde7631d0b3aa4e2ad3e318adbfcb94ed036ad71f1866383e" exitCode=0 Jul 16 05:27:37.323654 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:37.323295 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" event={"ID":"ab9072b8-695b-4832-965c-7bde95599464","Type":"ContainerDied","Data":"3143f8c04e7c290bde7631d0b3aa4e2ad3e318adbfcb94ed036ad71f1866383e"} Jul 16 05:27:37.323724 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:37.323665 2561 scope.go:117] "RemoveContainer" containerID="3143f8c04e7c290bde7631d0b3aa4e2ad3e318adbfcb94ed036ad71f1866383e" Jul 16 05:27:37.324655 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:37.324629 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:27:37.324770 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:37.324716 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:27:37.324770 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:37.324757 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:27:38.330629 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:38.330586 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" event={"ID":"ab9072b8-695b-4832-965c-7bde95599464","Type":"ContainerStarted","Data":"1ecee5a3af27aed66cdb53f27b56eb475d1761bc1b1991228a1ebb1c46350119"} Jul 16 05:27:38.979180 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:38.979134 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:38.979401 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:38.979278 2561 patch_prober.go:28] interesting pod/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Jul 16 05:27:38.979401 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:38.979345 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" podUID="ab9072b8-695b-4832-965c-7bde95599464" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Jul 16 05:27:39.241171 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:39.241087 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:39.978186 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:39.978146 2561 patch_prober.go:28] interesting pod/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Jul 16 05:27:39.978827 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:39.978776 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" podUID="ab9072b8-695b-4832-965c-7bde95599464" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Jul 16 05:27:40.977460 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:40.977422 2561 patch_prober.go:28] interesting pod/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Jul 16 05:27:40.977794 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:40.977760 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" podUID="ab9072b8-695b-4832-965c-7bde95599464" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Jul 16 05:27:41.977839 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:41.977799 2561 patch_prober.go:28] interesting pod/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Jul 16 05:27:41.978430 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:41.978398 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" podUID="ab9072b8-695b-4832-965c-7bde95599464" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Jul 16 05:27:42.981866 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:42.981834 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:27:42.983055 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:27:42.983031 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:29:40.347213 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:40.347171 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w"] Jul 16 05:29:40.347711 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:40.347393 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" podUID="ab9072b8-695b-4832-965c-7bde95599464" containerName="istio-proxy" containerID="cri-o://1ecee5a3af27aed66cdb53f27b56eb475d1761bc1b1991228a1ebb1c46350119" gracePeriod=30 Jul 16 05:29:42.984179 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:42.984133 2561 patch_prober.go:28] interesting pod/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 05:29:42.984567 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:42.984216 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" podUID="ab9072b8-695b-4832-965c-7bde95599464" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 05:29:45.592258 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.592228 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:29:45.704055 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.703962 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-istio-data\") pod \"ab9072b8-695b-4832-965c-7bde95599464\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " Jul 16 05:29:45.704055 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.704007 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-credential-socket\") pod \"ab9072b8-695b-4832-965c-7bde95599464\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " Jul 16 05:29:45.704055 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.704037 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab9072b8-695b-4832-965c-7bde95599464-istio-podinfo\") pod \"ab9072b8-695b-4832-965c-7bde95599464\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " Jul 16 05:29:45.704055 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.704051 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-workload-socket\") pod \"ab9072b8-695b-4832-965c-7bde95599464\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " Jul 16 05:29:45.704397 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.704070 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab9072b8-695b-4832-965c-7bde95599464-istiod-ca-cert\") pod \"ab9072b8-695b-4832-965c-7bde95599464\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " Jul 16 05:29:45.704397 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.704127 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-istio-envoy\") pod \"ab9072b8-695b-4832-965c-7bde95599464\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " Jul 16 05:29:45.704397 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.704154 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab9072b8-695b-4832-965c-7bde95599464-istio-token\") pod \"ab9072b8-695b-4832-965c-7bde95599464\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " Jul 16 05:29:45.704397 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.704197 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cx6zg\" (UniqueName: \"kubernetes.io/projected/ab9072b8-695b-4832-965c-7bde95599464-kube-api-access-cx6zg\") pod \"ab9072b8-695b-4832-965c-7bde95599464\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " Jul 16 05:29:45.704397 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.704299 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-workload-certs\") pod \"ab9072b8-695b-4832-965c-7bde95599464\" (UID: \"ab9072b8-695b-4832-965c-7bde95599464\") " Jul 16 05:29:45.704772 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.704321 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ab9072b8-695b-4832-965c-7bde95599464" (UID: "ab9072b8-695b-4832-965c-7bde95599464"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:29:45.704943 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.704351 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ab9072b8-695b-4832-965c-7bde95599464" (UID: "ab9072b8-695b-4832-965c-7bde95599464"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:29:45.705292 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.704679 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab9072b8-695b-4832-965c-7bde95599464-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ab9072b8-695b-4832-965c-7bde95599464" (UID: "ab9072b8-695b-4832-965c-7bde95599464"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 05:29:45.705292 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.705176 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-istio-data" (OuterVolumeSpecName: "istio-data") pod "ab9072b8-695b-4832-965c-7bde95599464" (UID: "ab9072b8-695b-4832-965c-7bde95599464"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:29:45.705292 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.705229 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ab9072b8-695b-4832-965c-7bde95599464" (UID: "ab9072b8-695b-4832-965c-7bde95599464"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:29:45.706543 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.706518 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ab9072b8-695b-4832-965c-7bde95599464" (UID: "ab9072b8-695b-4832-965c-7bde95599464"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:29:45.706543 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.706542 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab9072b8-695b-4832-965c-7bde95599464-istio-token" (OuterVolumeSpecName: "istio-token") pod "ab9072b8-695b-4832-965c-7bde95599464" (UID: "ab9072b8-695b-4832-965c-7bde95599464"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:29:45.706708 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.706585 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab9072b8-695b-4832-965c-7bde95599464-kube-api-access-cx6zg" (OuterVolumeSpecName: "kube-api-access-cx6zg") pod "ab9072b8-695b-4832-965c-7bde95599464" (UID: "ab9072b8-695b-4832-965c-7bde95599464"). InnerVolumeSpecName "kube-api-access-cx6zg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:29:45.706708 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.706599 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ab9072b8-695b-4832-965c-7bde95599464-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ab9072b8-695b-4832-965c-7bde95599464" (UID: "ab9072b8-695b-4832-965c-7bde95599464"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 05:29:45.806090 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.806058 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-istio-data\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:29:45.806090 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.806090 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-credential-socket\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:29:45.806090 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.806101 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab9072b8-695b-4832-965c-7bde95599464-istio-podinfo\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:29:45.806333 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.806111 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-workload-socket\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:29:45.806333 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.806120 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab9072b8-695b-4832-965c-7bde95599464-istiod-ca-cert\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:29:45.806333 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.806127 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-istio-envoy\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:29:45.806333 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.806134 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab9072b8-695b-4832-965c-7bde95599464-istio-token\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:29:45.806333 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.806142 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cx6zg\" (UniqueName: \"kubernetes.io/projected/ab9072b8-695b-4832-965c-7bde95599464-kube-api-access-cx6zg\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:29:45.806333 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.806150 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab9072b8-695b-4832-965c-7bde95599464-workload-certs\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:29:45.882587 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.882556 2561 generic.go:356] "Generic (PLEG): container finished" podID="ab9072b8-695b-4832-965c-7bde95599464" containerID="1ecee5a3af27aed66cdb53f27b56eb475d1761bc1b1991228a1ebb1c46350119" exitCode=0 Jul 16 05:29:45.882769 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.882609 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" Jul 16 05:29:45.882769 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.882605 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" event={"ID":"ab9072b8-695b-4832-965c-7bde95599464","Type":"ContainerDied","Data":"1ecee5a3af27aed66cdb53f27b56eb475d1761bc1b1991228a1ebb1c46350119"} Jul 16 05:29:45.882769 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.882761 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w" event={"ID":"ab9072b8-695b-4832-965c-7bde95599464","Type":"ContainerDied","Data":"3b9f31c3465745fa55edce75c20d1242100934093fffccc46059c37a33700121"} Jul 16 05:29:45.882907 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.882787 2561 scope.go:117] "RemoveContainer" containerID="1ecee5a3af27aed66cdb53f27b56eb475d1761bc1b1991228a1ebb1c46350119" Jul 16 05:29:45.891976 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.891962 2561 scope.go:117] "RemoveContainer" containerID="3143f8c04e7c290bde7631d0b3aa4e2ad3e318adbfcb94ed036ad71f1866383e" Jul 16 05:29:45.899854 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.899833 2561 scope.go:117] "RemoveContainer" containerID="1ecee5a3af27aed66cdb53f27b56eb475d1761bc1b1991228a1ebb1c46350119" Jul 16 05:29:45.900108 ip-10-0-135-187 kubenswrapper[2561]: E0716 05:29:45.900086 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ecee5a3af27aed66cdb53f27b56eb475d1761bc1b1991228a1ebb1c46350119\": container with ID starting with 1ecee5a3af27aed66cdb53f27b56eb475d1761bc1b1991228a1ebb1c46350119 not found: ID does not exist" containerID="1ecee5a3af27aed66cdb53f27b56eb475d1761bc1b1991228a1ebb1c46350119" Jul 16 05:29:45.900161 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.900115 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ecee5a3af27aed66cdb53f27b56eb475d1761bc1b1991228a1ebb1c46350119"} err="failed to get container status \"1ecee5a3af27aed66cdb53f27b56eb475d1761bc1b1991228a1ebb1c46350119\": rpc error: code = NotFound desc = could not find container \"1ecee5a3af27aed66cdb53f27b56eb475d1761bc1b1991228a1ebb1c46350119\": container with ID starting with 1ecee5a3af27aed66cdb53f27b56eb475d1761bc1b1991228a1ebb1c46350119 not found: ID does not exist" Jul 16 05:29:45.900161 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.900134 2561 scope.go:117] "RemoveContainer" containerID="3143f8c04e7c290bde7631d0b3aa4e2ad3e318adbfcb94ed036ad71f1866383e" Jul 16 05:29:45.900364 ip-10-0-135-187 kubenswrapper[2561]: E0716 05:29:45.900345 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3143f8c04e7c290bde7631d0b3aa4e2ad3e318adbfcb94ed036ad71f1866383e\": container with ID starting with 3143f8c04e7c290bde7631d0b3aa4e2ad3e318adbfcb94ed036ad71f1866383e not found: ID does not exist" containerID="3143f8c04e7c290bde7631d0b3aa4e2ad3e318adbfcb94ed036ad71f1866383e" Jul 16 05:29:45.900409 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.900371 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3143f8c04e7c290bde7631d0b3aa4e2ad3e318adbfcb94ed036ad71f1866383e"} err="failed to get container status \"3143f8c04e7c290bde7631d0b3aa4e2ad3e318adbfcb94ed036ad71f1866383e\": rpc error: code = NotFound desc = could not find container \"3143f8c04e7c290bde7631d0b3aa4e2ad3e318adbfcb94ed036ad71f1866383e\": container with ID starting with 3143f8c04e7c290bde7631d0b3aa4e2ad3e318adbfcb94ed036ad71f1866383e not found: ID does not exist" Jul 16 05:29:45.904369 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.904349 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w"] Jul 16 05:29:45.909852 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:45.909821 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-18c409ba-openshift-default-dcdcdfb55-9k52w"] Jul 16 05:29:46.100519 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:29:46.100487 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab9072b8-695b-4832-965c-7bde95599464" path="/var/lib/kubelet/pods/ab9072b8-695b-4832-965c-7bde95599464/volumes" Jul 16 05:30:35.079746 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.079714 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k"] Jul 16 05:30:35.080142 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.080017 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dbb07fa-ca64-495c-aa34-f978d3f0b175" containerName="istio-proxy" Jul 16 05:30:35.080142 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.080028 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dbb07fa-ca64-495c-aa34-f978d3f0b175" containerName="istio-proxy" Jul 16 05:30:35.080142 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.080037 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab9072b8-695b-4832-965c-7bde95599464" containerName="istio-proxy" Jul 16 05:30:35.080142 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.080043 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab9072b8-695b-4832-965c-7bde95599464" containerName="istio-proxy" Jul 16 05:30:35.080142 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.080061 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab9072b8-695b-4832-965c-7bde95599464" containerName="istio-proxy" Jul 16 05:30:35.080142 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.080066 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab9072b8-695b-4832-965c-7bde95599464" containerName="istio-proxy" Jul 16 05:30:35.080142 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.080115 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab9072b8-695b-4832-965c-7bde95599464" containerName="istio-proxy" Jul 16 05:30:35.080142 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.080123 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab9072b8-695b-4832-965c-7bde95599464" containerName="istio-proxy" Jul 16 05:30:35.083022 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.083005 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.085617 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.085593 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-a956af-openshift-default-dockercfg-2fchg\"" Jul 16 05:30:35.095746 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.095720 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k"] Jul 16 05:30:35.223961 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.223927 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p86l9\" (UniqueName: \"kubernetes.io/projected/74741281-195a-4f28-b355-5767a188d4dd-kube-api-access-p86l9\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.223961 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.223967 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-istio-envoy\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.224211 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.223990 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74741281-195a-4f28-b355-5767a188d4dd-istiod-ca-cert\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.224211 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.224008 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-workload-socket\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.224211 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.224063 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-credential-socket\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.224211 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.224106 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74741281-195a-4f28-b355-5767a188d4dd-istio-token\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.224211 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.224144 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-workload-certs\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.224211 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.224159 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-istio-data\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.224456 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.224240 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74741281-195a-4f28-b355-5767a188d4dd-istio-podinfo\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.324858 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.324820 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p86l9\" (UniqueName: \"kubernetes.io/projected/74741281-195a-4f28-b355-5767a188d4dd-kube-api-access-p86l9\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.325033 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.324901 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-istio-envoy\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.325033 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.324958 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74741281-195a-4f28-b355-5767a188d4dd-istiod-ca-cert\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.325033 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.324980 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-workload-socket\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.325033 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.325001 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-credential-socket\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.325033 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.325016 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74741281-195a-4f28-b355-5767a188d4dd-istio-token\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.325288 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.325065 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-workload-certs\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.325288 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.325090 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-istio-data\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.325288 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.325133 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74741281-195a-4f28-b355-5767a188d4dd-istio-podinfo\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.325431 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.325356 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-workload-socket\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.325431 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.325414 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-credential-socket\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.325515 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.325499 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-istio-data\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.325609 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.325587 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-workload-certs\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.326112 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.326088 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74741281-195a-4f28-b355-5767a188d4dd-istiod-ca-cert\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.327354 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.327327 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-istio-envoy\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.327702 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.327669 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74741281-195a-4f28-b355-5767a188d4dd-istio-podinfo\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.332470 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.332420 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p86l9\" (UniqueName: \"kubernetes.io/projected/74741281-195a-4f28-b355-5767a188d4dd-kube-api-access-p86l9\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.332470 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.332473 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74741281-195a-4f28-b355-5767a188d4dd-istio-token\") pod \"e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.393824 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.393790 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:35.528352 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.528329 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k"] Jul 16 05:30:35.530534 ip-10-0-135-187 kubenswrapper[2561]: W0716 05:30:35.530504 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74741281_195a_4f28_b355_5767a188d4dd.slice/crio-76253dd8713435fc396f329111f26c64bffa9a4117175c6bb77767b4f594ce25 WatchSource:0}: Error finding container 76253dd8713435fc396f329111f26c64bffa9a4117175c6bb77767b4f594ce25: Status 404 returned error can't find the container with id 76253dd8713435fc396f329111f26c64bffa9a4117175c6bb77767b4f594ce25 Jul 16 05:30:35.532575 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.532544 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:30:35.532653 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.532607 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:30:35.532653 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:35.532635 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:30:36.102645 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:36.102606 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" event={"ID":"74741281-195a-4f28-b355-5767a188d4dd","Type":"ContainerStarted","Data":"2be99d1952e9fc85b3b312e683c3f4e43680848c4d10466bf150ec0e3da271a2"} Jul 16 05:30:36.103056 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:36.102662 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" event={"ID":"74741281-195a-4f28-b355-5767a188d4dd","Type":"ContainerStarted","Data":"76253dd8713435fc396f329111f26c64bffa9a4117175c6bb77767b4f594ce25"} Jul 16 05:30:36.122393 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:36.122343 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" podStartSLOduration=1.122328873 podStartE2EDuration="1.122328873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:30:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:30:36.120170309 +0000 UTC m=+2426.492293532" watchObservedRunningTime="2026-07-16 05:30:36.122328873 +0000 UTC m=+2426.494452077" Jul 16 05:30:36.394979 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:36.394890 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:36.399889 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:36.399863 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:37.107363 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:37.107331 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:37.108288 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:37.108266 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:43.380907 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.380872 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd"] Jul 16 05:30:43.389739 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.389710 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.392156 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.392133 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-309054-openshift-default-dockercfg-tmmtn\"" Jul 16 05:30:43.396136 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.396114 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd"] Jul 16 05:30:43.494744 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.494674 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-workload-certs\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.495020 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.495000 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e937ad06-781d-4e47-85c1-0b50043c3ccf-istio-podinfo\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.495175 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.495159 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-252l4\" (UniqueName: \"kubernetes.io/projected/e937ad06-781d-4e47-85c1-0b50043c3ccf-kube-api-access-252l4\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.495309 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.495291 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-workload-socket\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.495423 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.495405 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-istio-data\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.495531 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.495517 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e937ad06-781d-4e47-85c1-0b50043c3ccf-istio-token\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.495700 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.495667 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-credential-socket\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.495823 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.495770 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-istio-envoy\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.495881 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.495823 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e937ad06-781d-4e47-85c1-0b50043c3ccf-istiod-ca-cert\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.597105 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.597069 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-credential-socket\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.597268 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.597115 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-istio-envoy\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.597268 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.597139 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e937ad06-781d-4e47-85c1-0b50043c3ccf-istiod-ca-cert\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.597268 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.597175 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-workload-certs\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.597268 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.597198 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e937ad06-781d-4e47-85c1-0b50043c3ccf-istio-podinfo\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.597268 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.597223 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-252l4\" (UniqueName: \"kubernetes.io/projected/e937ad06-781d-4e47-85c1-0b50043c3ccf-kube-api-access-252l4\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.597268 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.597240 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-workload-socket\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.597268 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.597258 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-istio-data\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.597651 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.597279 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e937ad06-781d-4e47-85c1-0b50043c3ccf-istio-token\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.597651 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.597577 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-credential-socket\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.597783 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.597652 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-workload-certs\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.597828 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.597783 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-istio-data\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.598025 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.598006 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e937ad06-781d-4e47-85c1-0b50043c3ccf-istiod-ca-cert\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.598065 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.598010 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-workload-socket\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.600176 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.600156 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-istio-envoy\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.600424 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.600404 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e937ad06-781d-4e47-85c1-0b50043c3ccf-istio-podinfo\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.605141 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.605116 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-252l4\" (UniqueName: \"kubernetes.io/projected/e937ad06-781d-4e47-85c1-0b50043c3ccf-kube-api-access-252l4\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.605227 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.605207 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e937ad06-781d-4e47-85c1-0b50043c3ccf-istio-token\") pod \"e2e-inf-b-309054-openshift-default-5544f5989c-cztsd\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.701713 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.701604 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:43.836802 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.836767 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd"] Jul 16 05:30:43.838627 ip-10-0-135-187 kubenswrapper[2561]: W0716 05:30:43.838594 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode937ad06_781d_4e47_85c1_0b50043c3ccf.slice/crio-56763f220c722a5ca5c45e080f531577eb1f27d4c341cb622fa4bf0d7a23bb61 WatchSource:0}: Error finding container 56763f220c722a5ca5c45e080f531577eb1f27d4c341cb622fa4bf0d7a23bb61: Status 404 returned error can't find the container with id 56763f220c722a5ca5c45e080f531577eb1f27d4c341cb622fa4bf0d7a23bb61 Jul 16 05:30:43.840968 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.840938 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:30:43.841034 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.841008 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:30:43.841082 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:43.841035 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:30:44.138396 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:44.138356 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" event={"ID":"e937ad06-781d-4e47-85c1-0b50043c3ccf","Type":"ContainerStarted","Data":"6c3b67689a3551954ebccc5cac0ea3822ae3e76b42cc0a9b030377399ad359cc"} Jul 16 05:30:44.138607 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:44.138408 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" event={"ID":"e937ad06-781d-4e47-85c1-0b50043c3ccf","Type":"ContainerStarted","Data":"56763f220c722a5ca5c45e080f531577eb1f27d4c341cb622fa4bf0d7a23bb61"} Jul 16 05:30:44.158826 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:44.158775 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" podStartSLOduration=1.158759163 podStartE2EDuration="1.158759163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:30:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:30:44.157168585 +0000 UTC m=+2434.529291811" watchObservedRunningTime="2026-07-16 05:30:44.158759163 +0000 UTC m=+2434.530882424" Jul 16 05:30:44.702796 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:44.702754 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:44.707583 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:44.707555 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:45.143062 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:45.143031 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:45.144075 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:45.144056 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:30:53.148849 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:53.148808 2561 patch_prober.go:28] interesting pod/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 16 05:30:53.149231 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:53.148866 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" podUID="74741281-195a-4f28-b355-5767a188d4dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 16 05:30:55.193077 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:55.193035 2561 generic.go:356] "Generic (PLEG): container finished" podID="74741281-195a-4f28-b355-5767a188d4dd" containerID="2be99d1952e9fc85b3b312e683c3f4e43680848c4d10466bf150ec0e3da271a2" exitCode=0 Jul 16 05:30:55.193523 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:55.193115 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" event={"ID":"74741281-195a-4f28-b355-5767a188d4dd","Type":"ContainerDied","Data":"2be99d1952e9fc85b3b312e683c3f4e43680848c4d10466bf150ec0e3da271a2"} Jul 16 05:30:55.193605 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:55.193587 2561 scope.go:117] "RemoveContainer" containerID="2be99d1952e9fc85b3b312e683c3f4e43680848c4d10466bf150ec0e3da271a2" Jul 16 05:30:55.194611 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:55.194578 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:30:55.194741 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:55.194654 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:30:55.194741 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:55.194716 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:30:56.200816 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:56.200779 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" event={"ID":"74741281-195a-4f28-b355-5767a188d4dd","Type":"ContainerStarted","Data":"4586c3b493c2c75d70b1a32b671a8df7bbdb87d5761414eb9e90b0d891e21663"} Jul 16 05:30:56.394236 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:56.394173 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:30:56.394799 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:56.394766 2561 patch_prober.go:28] interesting pod/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 16 05:30:56.394943 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:56.394827 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" podUID="74741281-195a-4f28-b355-5767a188d4dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 16 05:30:57.395043 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:57.394994 2561 patch_prober.go:28] interesting pod/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 16 05:30:57.395568 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:57.395068 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" podUID="74741281-195a-4f28-b355-5767a188d4dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 16 05:30:58.394288 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:58.394247 2561 patch_prober.go:28] interesting pod/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 16 05:30:58.395054 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:58.394586 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" podUID="74741281-195a-4f28-b355-5767a188d4dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 16 05:30:59.394603 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:59.394549 2561 patch_prober.go:28] interesting pod/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 16 05:30:59.395135 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:30:59.394625 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" podUID="74741281-195a-4f28-b355-5767a188d4dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 16 05:31:00.394464 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:00.394414 2561 patch_prober.go:28] interesting pod/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 16 05:31:00.394678 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:00.394501 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" podUID="74741281-195a-4f28-b355-5767a188d4dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 16 05:31:01.184886 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:01.184841 2561 patch_prober.go:28] interesting pod/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 16 05:31:01.185080 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:01.184929 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" podUID="e937ad06-781d-4e47-85c1-0b50043c3ccf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 16 05:31:01.398236 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:01.398205 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:31:01.398710 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:01.398614 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:31:01.399223 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:01.399205 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:31:02.237658 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:02.237559 2561 generic.go:356] "Generic (PLEG): container finished" podID="e937ad06-781d-4e47-85c1-0b50043c3ccf" containerID="6c3b67689a3551954ebccc5cac0ea3822ae3e76b42cc0a9b030377399ad359cc" exitCode=0 Jul 16 05:31:02.237658 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:02.237633 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" event={"ID":"e937ad06-781d-4e47-85c1-0b50043c3ccf","Type":"ContainerDied","Data":"6c3b67689a3551954ebccc5cac0ea3822ae3e76b42cc0a9b030377399ad359cc"} Jul 16 05:31:02.238146 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:02.238129 2561 scope.go:117] "RemoveContainer" containerID="6c3b67689a3551954ebccc5cac0ea3822ae3e76b42cc0a9b030377399ad359cc" Jul 16 05:31:02.239149 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:02.239122 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:31:02.239222 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:02.239205 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:31:02.239283 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:02.239268 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:31:03.244660 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:03.244621 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" event={"ID":"e937ad06-781d-4e47-85c1-0b50043c3ccf","Type":"ContainerStarted","Data":"e7dbacaf8da0ed4ce31c28b755e44ba895219e848181e389c80ab371866a73b9"} Jul 16 05:31:03.702778 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:03.702726 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:31:03.703035 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:03.703011 2561 patch_prober.go:28] interesting pod/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 16 05:31:03.703124 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:03.703070 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" podUID="e937ad06-781d-4e47-85c1-0b50043c3ccf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 16 05:31:04.702716 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:04.702667 2561 patch_prober.go:28] interesting pod/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 16 05:31:04.703098 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:04.702737 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" podUID="e937ad06-781d-4e47-85c1-0b50043c3ccf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 16 05:31:05.702993 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:05.702958 2561 patch_prober.go:28] interesting pod/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 16 05:31:05.703394 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:05.703018 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" podUID="e937ad06-781d-4e47-85c1-0b50043c3ccf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 16 05:31:06.702117 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:06.702070 2561 patch_prober.go:28] interesting pod/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 16 05:31:06.702314 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:06.702158 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" podUID="e937ad06-781d-4e47-85c1-0b50043c3ccf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 16 05:31:07.706317 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:07.706276 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:31:07.706806 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:07.706698 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:31:07.707425 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:31:07.707408 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:32:11.484038 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:11.484004 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k"] Jul 16 05:32:11.484519 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:11.484296 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" podUID="74741281-195a-4f28-b355-5767a188d4dd" containerName="istio-proxy" containerID="cri-o://4586c3b493c2c75d70b1a32b671a8df7bbdb87d5761414eb9e90b0d891e21663" gracePeriod=30 Jul 16 05:32:16.401152 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.401118 2561 patch_prober.go:28] interesting pod/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 05:32:16.401535 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.401194 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" podUID="74741281-195a-4f28-b355-5767a188d4dd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 05:32:16.737163 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.737137 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:32:16.856361 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.856321 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p86l9\" (UniqueName: \"kubernetes.io/projected/74741281-195a-4f28-b355-5767a188d4dd-kube-api-access-p86l9\") pod \"74741281-195a-4f28-b355-5767a188d4dd\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " Jul 16 05:32:16.856562 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.856368 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-credential-socket\") pod \"74741281-195a-4f28-b355-5767a188d4dd\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " Jul 16 05:32:16.856562 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.856402 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74741281-195a-4f28-b355-5767a188d4dd-istio-token\") pod \"74741281-195a-4f28-b355-5767a188d4dd\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " Jul 16 05:32:16.856562 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.856425 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-workload-certs\") pod \"74741281-195a-4f28-b355-5767a188d4dd\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " Jul 16 05:32:16.856562 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.856462 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74741281-195a-4f28-b355-5767a188d4dd-istio-podinfo\") pod \"74741281-195a-4f28-b355-5767a188d4dd\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " Jul 16 05:32:16.856562 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.856513 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74741281-195a-4f28-b355-5767a188d4dd-istiod-ca-cert\") pod \"74741281-195a-4f28-b355-5767a188d4dd\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " Jul 16 05:32:16.856562 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.856548 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-istio-data\") pod \"74741281-195a-4f28-b355-5767a188d4dd\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " Jul 16 05:32:16.856914 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.856632 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-workload-socket\") pod \"74741281-195a-4f28-b355-5767a188d4dd\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " Jul 16 05:32:16.856914 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.856671 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-istio-envoy\") pod \"74741281-195a-4f28-b355-5767a188d4dd\" (UID: \"74741281-195a-4f28-b355-5767a188d4dd\") " Jul 16 05:32:16.856914 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.856756 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "74741281-195a-4f28-b355-5767a188d4dd" (UID: "74741281-195a-4f28-b355-5767a188d4dd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:32:16.856914 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.856855 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "74741281-195a-4f28-b355-5767a188d4dd" (UID: "74741281-195a-4f28-b355-5767a188d4dd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:32:16.856914 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.856905 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74741281-195a-4f28-b355-5767a188d4dd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "74741281-195a-4f28-b355-5767a188d4dd" (UID: "74741281-195a-4f28-b355-5767a188d4dd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 05:32:16.857187 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.857039 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74741281-195a-4f28-b355-5767a188d4dd-istiod-ca-cert\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:32:16.857187 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.857072 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-credential-socket\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:32:16.857187 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.857088 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-workload-certs\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:32:16.857187 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.857042 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "74741281-195a-4f28-b355-5767a188d4dd" (UID: "74741281-195a-4f28-b355-5767a188d4dd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:32:16.857950 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.857919 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-istio-data" (OuterVolumeSpecName: "istio-data") pod "74741281-195a-4f28-b355-5767a188d4dd" (UID: "74741281-195a-4f28-b355-5767a188d4dd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:32:16.858905 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.858885 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74741281-195a-4f28-b355-5767a188d4dd-istio-token" (OuterVolumeSpecName: "istio-token") pod "74741281-195a-4f28-b355-5767a188d4dd" (UID: "74741281-195a-4f28-b355-5767a188d4dd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:32:16.859031 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.859018 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "74741281-195a-4f28-b355-5767a188d4dd" (UID: "74741281-195a-4f28-b355-5767a188d4dd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:32:16.859198 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.859184 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/74741281-195a-4f28-b355-5767a188d4dd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "74741281-195a-4f28-b355-5767a188d4dd" (UID: "74741281-195a-4f28-b355-5767a188d4dd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 05:32:16.859256 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.859203 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74741281-195a-4f28-b355-5767a188d4dd-kube-api-access-p86l9" (OuterVolumeSpecName: "kube-api-access-p86l9") pod "74741281-195a-4f28-b355-5767a188d4dd" (UID: "74741281-195a-4f28-b355-5767a188d4dd"). InnerVolumeSpecName "kube-api-access-p86l9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:32:16.957972 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.957932 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-workload-socket\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:32:16.957972 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.957967 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-istio-envoy\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:32:16.957972 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.957977 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p86l9\" (UniqueName: \"kubernetes.io/projected/74741281-195a-4f28-b355-5767a188d4dd-kube-api-access-p86l9\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:32:16.958205 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.957987 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74741281-195a-4f28-b355-5767a188d4dd-istio-token\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:32:16.958205 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.957995 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74741281-195a-4f28-b355-5767a188d4dd-istio-podinfo\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:32:16.958205 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:16.958002 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74741281-195a-4f28-b355-5767a188d4dd-istio-data\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:32:17.584067 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:17.584035 2561 generic.go:356] "Generic (PLEG): container finished" podID="74741281-195a-4f28-b355-5767a188d4dd" containerID="4586c3b493c2c75d70b1a32b671a8df7bbdb87d5761414eb9e90b0d891e21663" exitCode=0 Jul 16 05:32:17.584486 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:17.584073 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" event={"ID":"74741281-195a-4f28-b355-5767a188d4dd","Type":"ContainerDied","Data":"4586c3b493c2c75d70b1a32b671a8df7bbdb87d5761414eb9e90b0d891e21663"} Jul 16 05:32:17.584486 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:17.584102 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" Jul 16 05:32:17.584486 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:17.584115 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k" event={"ID":"74741281-195a-4f28-b355-5767a188d4dd","Type":"ContainerDied","Data":"76253dd8713435fc396f329111f26c64bffa9a4117175c6bb77767b4f594ce25"} Jul 16 05:32:17.584486 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:17.584136 2561 scope.go:117] "RemoveContainer" containerID="4586c3b493c2c75d70b1a32b671a8df7bbdb87d5761414eb9e90b0d891e21663" Jul 16 05:32:17.593287 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:17.593269 2561 scope.go:117] "RemoveContainer" containerID="2be99d1952e9fc85b3b312e683c3f4e43680848c4d10466bf150ec0e3da271a2" Jul 16 05:32:17.600963 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:17.600807 2561 scope.go:117] "RemoveContainer" containerID="4586c3b493c2c75d70b1a32b671a8df7bbdb87d5761414eb9e90b0d891e21663" Jul 16 05:32:17.601095 ip-10-0-135-187 kubenswrapper[2561]: E0716 05:32:17.601076 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4586c3b493c2c75d70b1a32b671a8df7bbdb87d5761414eb9e90b0d891e21663\": container with ID starting with 4586c3b493c2c75d70b1a32b671a8df7bbdb87d5761414eb9e90b0d891e21663 not found: ID does not exist" containerID="4586c3b493c2c75d70b1a32b671a8df7bbdb87d5761414eb9e90b0d891e21663" Jul 16 05:32:17.601171 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:17.601107 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4586c3b493c2c75d70b1a32b671a8df7bbdb87d5761414eb9e90b0d891e21663"} err="failed to get container status \"4586c3b493c2c75d70b1a32b671a8df7bbdb87d5761414eb9e90b0d891e21663\": rpc error: code = NotFound desc = could not find container \"4586c3b493c2c75d70b1a32b671a8df7bbdb87d5761414eb9e90b0d891e21663\": container with ID starting with 4586c3b493c2c75d70b1a32b671a8df7bbdb87d5761414eb9e90b0d891e21663 not found: ID does not exist" Jul 16 05:32:17.601171 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:17.601132 2561 scope.go:117] "RemoveContainer" containerID="2be99d1952e9fc85b3b312e683c3f4e43680848c4d10466bf150ec0e3da271a2" Jul 16 05:32:17.601387 ip-10-0-135-187 kubenswrapper[2561]: E0716 05:32:17.601370 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2be99d1952e9fc85b3b312e683c3f4e43680848c4d10466bf150ec0e3da271a2\": container with ID starting with 2be99d1952e9fc85b3b312e683c3f4e43680848c4d10466bf150ec0e3da271a2 not found: ID does not exist" containerID="2be99d1952e9fc85b3b312e683c3f4e43680848c4d10466bf150ec0e3da271a2" Jul 16 05:32:17.601427 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:17.601393 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2be99d1952e9fc85b3b312e683c3f4e43680848c4d10466bf150ec0e3da271a2"} err="failed to get container status \"2be99d1952e9fc85b3b312e683c3f4e43680848c4d10466bf150ec0e3da271a2\": rpc error: code = NotFound desc = could not find container \"2be99d1952e9fc85b3b312e683c3f4e43680848c4d10466bf150ec0e3da271a2\": container with ID starting with 2be99d1952e9fc85b3b312e683c3f4e43680848c4d10466bf150ec0e3da271a2 not found: ID does not exist" Jul 16 05:32:17.605286 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:17.605265 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k"] Jul 16 05:32:17.608438 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:17.608414 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-a956af-openshift-default-58678d896f-6sl2k"] Jul 16 05:32:18.101360 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:18.101327 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74741281-195a-4f28-b355-5767a188d4dd" path="/var/lib/kubelet/pods/74741281-195a-4f28-b355-5767a188d4dd/volumes" Jul 16 05:32:37.669352 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:37.669314 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd"] Jul 16 05:32:37.669872 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:37.669588 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" podUID="e937ad06-781d-4e47-85c1-0b50043c3ccf" containerName="istio-proxy" containerID="cri-o://e7dbacaf8da0ed4ce31c28b755e44ba895219e848181e389c80ab371866a73b9" gracePeriod=30 Jul 16 05:32:37.708490 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:37.708451 2561 patch_prober.go:28] interesting pod/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 05:32:37.708622 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:37.708513 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" podUID="e937ad06-781d-4e47-85c1-0b50043c3ccf" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 05:32:38.620582 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.620551 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc"] Jul 16 05:32:38.620888 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.620876 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74741281-195a-4f28-b355-5767a188d4dd" containerName="istio-proxy" Jul 16 05:32:38.620954 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.620889 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="74741281-195a-4f28-b355-5767a188d4dd" containerName="istio-proxy" Jul 16 05:32:38.620995 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.620958 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="74741281-195a-4f28-b355-5767a188d4dd" containerName="istio-proxy" Jul 16 05:32:38.624283 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.624259 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.626790 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.626769 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-ddd4fb-openshift-default-dockercfg-r62tp\"" Jul 16 05:32:38.635063 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.635041 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc"] Jul 16 05:32:38.753279 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.753247 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-credential-socket\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.753777 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.753295 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28a4248f-5aab-4b95-9b69-16a227d57738-istio-podinfo\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.753777 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.753401 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28a4248f-5aab-4b95-9b69-16a227d57738-istio-token\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.753777 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.753435 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-workload-socket\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.753777 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.753468 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-workload-certs\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.753777 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.753529 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-istio-data\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.753777 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.753564 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28a4248f-5aab-4b95-9b69-16a227d57738-istiod-ca-cert\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.753777 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.753593 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mj4g\" (UniqueName: \"kubernetes.io/projected/28a4248f-5aab-4b95-9b69-16a227d57738-kube-api-access-4mj4g\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.753777 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.753721 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-istio-envoy\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.854632 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.854601 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28a4248f-5aab-4b95-9b69-16a227d57738-istio-token\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.854865 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.854647 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-workload-socket\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.854865 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.854698 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-workload-certs\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.854865 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.854786 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-istio-data\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.854865 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.854828 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28a4248f-5aab-4b95-9b69-16a227d57738-istiod-ca-cert\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.855125 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.854964 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mj4g\" (UniqueName: \"kubernetes.io/projected/28a4248f-5aab-4b95-9b69-16a227d57738-kube-api-access-4mj4g\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.855125 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.855046 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-istio-envoy\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.855125 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.855081 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-credential-socket\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.855125 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.855087 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-workload-certs\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.855125 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.855113 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28a4248f-5aab-4b95-9b69-16a227d57738-istio-podinfo\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.855331 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.855122 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-workload-socket\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.855431 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.855390 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-credential-socket\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.855431 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.855413 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28a4248f-5aab-4b95-9b69-16a227d57738-istiod-ca-cert\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.855583 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.855529 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-istio-data\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.857389 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.857369 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-istio-envoy\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.857588 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.857570 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28a4248f-5aab-4b95-9b69-16a227d57738-istio-podinfo\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.862232 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.862199 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28a4248f-5aab-4b95-9b69-16a227d57738-istio-token\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.862348 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.862264 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mj4g\" (UniqueName: \"kubernetes.io/projected/28a4248f-5aab-4b95-9b69-16a227d57738-kube-api-access-4mj4g\") pod \"e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:38.935659 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:38.935580 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:39.072850 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:39.072822 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc"] Jul 16 05:32:39.075046 ip-10-0-135-187 kubenswrapper[2561]: W0716 05:32:39.075020 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28a4248f_5aab_4b95_9b69_16a227d57738.slice/crio-b4eed80a337008fe6f615c83b173ed24225efc7e6ea0246aeffb98e6b2f0dfd6 WatchSource:0}: Error finding container b4eed80a337008fe6f615c83b173ed24225efc7e6ea0246aeffb98e6b2f0dfd6: Status 404 returned error can't find the container with id b4eed80a337008fe6f615c83b173ed24225efc7e6ea0246aeffb98e6b2f0dfd6 Jul 16 05:32:39.077195 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:39.077164 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:32:39.077300 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:39.077242 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:32:39.077300 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:39.077289 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:32:39.685047 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:39.685011 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" event={"ID":"28a4248f-5aab-4b95-9b69-16a227d57738","Type":"ContainerStarted","Data":"53a12693a078da81ba2a4ad6f42c85611b411b2fa3e6bce8fff46245e0347c02"} Jul 16 05:32:39.685223 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:39.685083 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" event={"ID":"28a4248f-5aab-4b95-9b69-16a227d57738","Type":"ContainerStarted","Data":"b4eed80a337008fe6f615c83b173ed24225efc7e6ea0246aeffb98e6b2f0dfd6"} Jul 16 05:32:39.704164 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:39.704112 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" podStartSLOduration=1.704096981 podStartE2EDuration="1.704096981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:32:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:32:39.702160047 +0000 UTC m=+2550.074283291" watchObservedRunningTime="2026-07-16 05:32:39.704096981 +0000 UTC m=+2550.076220187" Jul 16 05:32:39.936606 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:39.936519 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:40.941284 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:40.941256 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:41.693181 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:41.693143 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:41.694189 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:41.694169 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:42.923614 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:42.923592 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:32:42.989580 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:42.989504 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-istio-data\") pod \"e937ad06-781d-4e47-85c1-0b50043c3ccf\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " Jul 16 05:32:42.989580 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:42.989540 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-workload-certs\") pod \"e937ad06-781d-4e47-85c1-0b50043c3ccf\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " Jul 16 05:32:42.989580 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:42.989557 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-credential-socket\") pod \"e937ad06-781d-4e47-85c1-0b50043c3ccf\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " Jul 16 05:32:42.989872 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:42.989606 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e937ad06-781d-4e47-85c1-0b50043c3ccf-istio-podinfo\") pod \"e937ad06-781d-4e47-85c1-0b50043c3ccf\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " Jul 16 05:32:42.989872 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:42.989635 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-workload-socket\") pod \"e937ad06-781d-4e47-85c1-0b50043c3ccf\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " Jul 16 05:32:42.989872 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:42.989656 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e937ad06-781d-4e47-85c1-0b50043c3ccf-istio-token\") pod \"e937ad06-781d-4e47-85c1-0b50043c3ccf\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " Jul 16 05:32:42.989872 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:42.989726 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-252l4\" (UniqueName: \"kubernetes.io/projected/e937ad06-781d-4e47-85c1-0b50043c3ccf-kube-api-access-252l4\") pod \"e937ad06-781d-4e47-85c1-0b50043c3ccf\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " Jul 16 05:32:42.989872 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:42.989755 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-istio-envoy\") pod \"e937ad06-781d-4e47-85c1-0b50043c3ccf\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " Jul 16 05:32:42.989872 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:42.989781 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e937ad06-781d-4e47-85c1-0b50043c3ccf-istiod-ca-cert\") pod \"e937ad06-781d-4e47-85c1-0b50043c3ccf\" (UID: \"e937ad06-781d-4e47-85c1-0b50043c3ccf\") " Jul 16 05:32:42.990156 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:42.989875 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e937ad06-781d-4e47-85c1-0b50043c3ccf" (UID: "e937ad06-781d-4e47-85c1-0b50043c3ccf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:32:42.990156 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:42.990002 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e937ad06-781d-4e47-85c1-0b50043c3ccf" (UID: "e937ad06-781d-4e47-85c1-0b50043c3ccf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:32:42.990156 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:42.990019 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e937ad06-781d-4e47-85c1-0b50043c3ccf" (UID: "e937ad06-781d-4e47-85c1-0b50043c3ccf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:32:42.990300 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:42.990243 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e937ad06-781d-4e47-85c1-0b50043c3ccf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e937ad06-781d-4e47-85c1-0b50043c3ccf" (UID: "e937ad06-781d-4e47-85c1-0b50043c3ccf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 05:32:42.990300 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:42.990272 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-workload-certs\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:32:42.990300 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:42.990293 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-credential-socket\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:32:42.990412 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:42.990310 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-workload-socket\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:32:42.990749 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:42.990725 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-istio-data" (OuterVolumeSpecName: "istio-data") pod "e937ad06-781d-4e47-85c1-0b50043c3ccf" (UID: "e937ad06-781d-4e47-85c1-0b50043c3ccf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:32:42.992065 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:42.992034 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e937ad06-781d-4e47-85c1-0b50043c3ccf-istio-token" (OuterVolumeSpecName: "istio-token") pod "e937ad06-781d-4e47-85c1-0b50043c3ccf" (UID: "e937ad06-781d-4e47-85c1-0b50043c3ccf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:32:42.992161 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:42.992061 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e937ad06-781d-4e47-85c1-0b50043c3ccf-kube-api-access-252l4" (OuterVolumeSpecName: "kube-api-access-252l4") pod "e937ad06-781d-4e47-85c1-0b50043c3ccf" (UID: "e937ad06-781d-4e47-85c1-0b50043c3ccf"). InnerVolumeSpecName "kube-api-access-252l4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:32:42.992535 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:42.992509 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e937ad06-781d-4e47-85c1-0b50043c3ccf" (UID: "e937ad06-781d-4e47-85c1-0b50043c3ccf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:32:42.992639 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:42.992509 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e937ad06-781d-4e47-85c1-0b50043c3ccf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e937ad06-781d-4e47-85c1-0b50043c3ccf" (UID: "e937ad06-781d-4e47-85c1-0b50043c3ccf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 05:32:43.090875 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:43.090837 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-istio-data\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:32:43.090875 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:43.090868 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e937ad06-781d-4e47-85c1-0b50043c3ccf-istio-podinfo\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:32:43.090875 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:43.090878 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e937ad06-781d-4e47-85c1-0b50043c3ccf-istio-token\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:32:43.090875 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:43.090887 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-252l4\" (UniqueName: \"kubernetes.io/projected/e937ad06-781d-4e47-85c1-0b50043c3ccf-kube-api-access-252l4\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:32:43.091145 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:43.090896 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e937ad06-781d-4e47-85c1-0b50043c3ccf-istio-envoy\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:32:43.091145 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:43.090904 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e937ad06-781d-4e47-85c1-0b50043c3ccf-istiod-ca-cert\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:32:43.706054 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:43.706015 2561 generic.go:356] "Generic (PLEG): container finished" podID="e937ad06-781d-4e47-85c1-0b50043c3ccf" containerID="e7dbacaf8da0ed4ce31c28b755e44ba895219e848181e389c80ab371866a73b9" exitCode=0 Jul 16 05:32:43.706247 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:43.706066 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" Jul 16 05:32:43.706247 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:43.706106 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" event={"ID":"e937ad06-781d-4e47-85c1-0b50043c3ccf","Type":"ContainerDied","Data":"e7dbacaf8da0ed4ce31c28b755e44ba895219e848181e389c80ab371866a73b9"} Jul 16 05:32:43.706247 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:43.706159 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd" event={"ID":"e937ad06-781d-4e47-85c1-0b50043c3ccf","Type":"ContainerDied","Data":"56763f220c722a5ca5c45e080f531577eb1f27d4c341cb622fa4bf0d7a23bb61"} Jul 16 05:32:43.706247 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:43.706184 2561 scope.go:117] "RemoveContainer" containerID="e7dbacaf8da0ed4ce31c28b755e44ba895219e848181e389c80ab371866a73b9" Jul 16 05:32:43.715419 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:43.715399 2561 scope.go:117] "RemoveContainer" containerID="6c3b67689a3551954ebccc5cac0ea3822ae3e76b42cc0a9b030377399ad359cc" Jul 16 05:32:43.723196 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:43.723178 2561 scope.go:117] "RemoveContainer" containerID="e7dbacaf8da0ed4ce31c28b755e44ba895219e848181e389c80ab371866a73b9" Jul 16 05:32:43.723444 ip-10-0-135-187 kubenswrapper[2561]: E0716 05:32:43.723424 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7dbacaf8da0ed4ce31c28b755e44ba895219e848181e389c80ab371866a73b9\": container with ID starting with e7dbacaf8da0ed4ce31c28b755e44ba895219e848181e389c80ab371866a73b9 not found: ID does not exist" containerID="e7dbacaf8da0ed4ce31c28b755e44ba895219e848181e389c80ab371866a73b9" Jul 16 05:32:43.723520 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:43.723457 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7dbacaf8da0ed4ce31c28b755e44ba895219e848181e389c80ab371866a73b9"} err="failed to get container status \"e7dbacaf8da0ed4ce31c28b755e44ba895219e848181e389c80ab371866a73b9\": rpc error: code = NotFound desc = could not find container \"e7dbacaf8da0ed4ce31c28b755e44ba895219e848181e389c80ab371866a73b9\": container with ID starting with e7dbacaf8da0ed4ce31c28b755e44ba895219e848181e389c80ab371866a73b9 not found: ID does not exist" Jul 16 05:32:43.723520 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:43.723490 2561 scope.go:117] "RemoveContainer" containerID="6c3b67689a3551954ebccc5cac0ea3822ae3e76b42cc0a9b030377399ad359cc" Jul 16 05:32:43.723772 ip-10-0-135-187 kubenswrapper[2561]: E0716 05:32:43.723754 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c3b67689a3551954ebccc5cac0ea3822ae3e76b42cc0a9b030377399ad359cc\": container with ID starting with 6c3b67689a3551954ebccc5cac0ea3822ae3e76b42cc0a9b030377399ad359cc not found: ID does not exist" containerID="6c3b67689a3551954ebccc5cac0ea3822ae3e76b42cc0a9b030377399ad359cc" Jul 16 05:32:43.723818 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:43.723779 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c3b67689a3551954ebccc5cac0ea3822ae3e76b42cc0a9b030377399ad359cc"} err="failed to get container status \"6c3b67689a3551954ebccc5cac0ea3822ae3e76b42cc0a9b030377399ad359cc\": rpc error: code = NotFound desc = could not find container \"6c3b67689a3551954ebccc5cac0ea3822ae3e76b42cc0a9b030377399ad359cc\": container with ID starting with 6c3b67689a3551954ebccc5cac0ea3822ae3e76b42cc0a9b030377399ad359cc not found: ID does not exist" Jul 16 05:32:43.726474 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:43.726454 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd"] Jul 16 05:32:43.733621 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:43.733600 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-309054-openshift-default-5544f5989c-cztsd"] Jul 16 05:32:44.100571 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:44.100539 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e937ad06-781d-4e47-85c1-0b50043c3ccf" path="/var/lib/kubelet/pods/e937ad06-781d-4e47-85c1-0b50043c3ccf/volumes" Jul 16 05:32:47.086229 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.086194 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb"] Jul 16 05:32:47.087269 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.087243 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e937ad06-781d-4e47-85c1-0b50043c3ccf" containerName="istio-proxy" Jul 16 05:32:47.087432 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.087419 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e937ad06-781d-4e47-85c1-0b50043c3ccf" containerName="istio-proxy" Jul 16 05:32:47.103826 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.103784 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74741281-195a-4f28-b355-5767a188d4dd" containerName="istio-proxy" Jul 16 05:32:47.104077 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.104058 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="74741281-195a-4f28-b355-5767a188d4dd" containerName="istio-proxy" Jul 16 05:32:47.104226 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.104213 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e937ad06-781d-4e47-85c1-0b50043c3ccf" containerName="istio-proxy" Jul 16 05:32:47.104319 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.104308 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e937ad06-781d-4e47-85c1-0b50043c3ccf" containerName="istio-proxy" Jul 16 05:32:47.104522 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.104508 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="74741281-195a-4f28-b355-5767a188d4dd" containerName="istio-proxy" Jul 16 05:32:47.104619 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.104609 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e937ad06-781d-4e47-85c1-0b50043c3ccf" containerName="istio-proxy" Jul 16 05:32:47.107636 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.107596 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.107771 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.107631 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb"] Jul 16 05:32:47.112025 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.112005 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-2797de-openshift-default-dockercfg-gdlfj\"" Jul 16 05:32:47.227191 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.227156 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-workload-certs\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.227374 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.227198 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-workload-socket\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.227374 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.227223 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23980ab8-c101-48e2-9a3f-d6e34bb85637-istiod-ca-cert\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.227374 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.227273 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvjww\" (UniqueName: \"kubernetes.io/projected/23980ab8-c101-48e2-9a3f-d6e34bb85637-kube-api-access-nvjww\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.227374 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.227312 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-credential-socket\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.227374 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.227350 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23980ab8-c101-48e2-9a3f-d6e34bb85637-istio-podinfo\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.227374 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.227372 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23980ab8-c101-48e2-9a3f-d6e34bb85637-istio-token\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.227746 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.227414 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-istio-data\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.227746 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.227450 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-istio-envoy\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.328475 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.328439 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvjww\" (UniqueName: \"kubernetes.io/projected/23980ab8-c101-48e2-9a3f-d6e34bb85637-kube-api-access-nvjww\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.328475 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.328491 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-credential-socket\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.328779 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.328514 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23980ab8-c101-48e2-9a3f-d6e34bb85637-istio-podinfo\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.328779 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.328540 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23980ab8-c101-48e2-9a3f-d6e34bb85637-istio-token\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.328779 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.328588 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-istio-data\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.328779 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.328623 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-istio-envoy\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.328779 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.328660 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-workload-certs\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.328779 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.328714 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-workload-socket\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.328779 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.328738 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23980ab8-c101-48e2-9a3f-d6e34bb85637-istiod-ca-cert\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.329127 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.328919 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-credential-socket\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.329127 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.328984 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-istio-data\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.329127 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.329057 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-workload-certs\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.329242 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.329205 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-workload-socket\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.329356 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.329336 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23980ab8-c101-48e2-9a3f-d6e34bb85637-istiod-ca-cert\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.331077 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.331062 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-istio-envoy\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.331314 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.331298 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23980ab8-c101-48e2-9a3f-d6e34bb85637-istio-podinfo\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.336420 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.336356 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23980ab8-c101-48e2-9a3f-d6e34bb85637-istio-token\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.336501 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.336474 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvjww\" (UniqueName: \"kubernetes.io/projected/23980ab8-c101-48e2-9a3f-d6e34bb85637-kube-api-access-nvjww\") pod \"e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.421015 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.420950 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:47.584711 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.577614 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb"] Jul 16 05:32:47.592827 ip-10-0-135-187 kubenswrapper[2561]: W0716 05:32:47.592741 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23980ab8_c101_48e2_9a3f_d6e34bb85637.slice/crio-7c7c2b22db892428cca5c41ca218fa58c4c3ee761fc0663a6ed9a48aaa2bf5d6 WatchSource:0}: Error finding container 7c7c2b22db892428cca5c41ca218fa58c4c3ee761fc0663a6ed9a48aaa2bf5d6: Status 404 returned error can't find the container with id 7c7c2b22db892428cca5c41ca218fa58c4c3ee761fc0663a6ed9a48aaa2bf5d6 Jul 16 05:32:47.596045 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.596013 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:32:47.596132 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.596087 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:32:47.596132 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.596118 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:32:47.726779 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.726729 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" event={"ID":"23980ab8-c101-48e2-9a3f-d6e34bb85637","Type":"ContainerStarted","Data":"21e2505c25b8d81c5e7c4b74d56820593d184263b1eb26f86c3da171413d2ef7"} Jul 16 05:32:47.726951 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.726830 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" event={"ID":"23980ab8-c101-48e2-9a3f-d6e34bb85637","Type":"ContainerStarted","Data":"7c7c2b22db892428cca5c41ca218fa58c4c3ee761fc0663a6ed9a48aaa2bf5d6"} Jul 16 05:32:47.744767 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:47.744710 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" podStartSLOduration=0.744659526 podStartE2EDuration="744.659526ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:32:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:32:47.744535081 +0000 UTC m=+2558.116658287" watchObservedRunningTime="2026-07-16 05:32:47.744659526 +0000 UTC m=+2558.116782735" Jul 16 05:32:48.421489 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:48.421447 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:48.426432 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:48.426406 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:48.732054 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:48.731962 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:48.733032 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:48.733011 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:32:54.760668 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:54.760636 2561 generic.go:356] "Generic (PLEG): container finished" podID="28a4248f-5aab-4b95-9b69-16a227d57738" containerID="53a12693a078da81ba2a4ad6f42c85611b411b2fa3e6bce8fff46245e0347c02" exitCode=0 Jul 16 05:32:54.761104 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:54.760713 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" event={"ID":"28a4248f-5aab-4b95-9b69-16a227d57738","Type":"ContainerDied","Data":"53a12693a078da81ba2a4ad6f42c85611b411b2fa3e6bce8fff46245e0347c02"} Jul 16 05:32:54.761149 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:54.761123 2561 scope.go:117] "RemoveContainer" containerID="53a12693a078da81ba2a4ad6f42c85611b411b2fa3e6bce8fff46245e0347c02" Jul 16 05:32:54.762109 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:54.762083 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:32:54.762234 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:54.762150 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:32:54.762234 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:54.762180 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:32:55.767854 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:55.767815 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" event={"ID":"28a4248f-5aab-4b95-9b69-16a227d57738","Type":"ContainerStarted","Data":"393fe4214e9c5a973e5b85d621fb5673885b2c3ea838e6babafa8ba2ba7431f5"} Jul 16 05:32:55.935883 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:55.935843 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:55.937090 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:55.937063 2561 patch_prober.go:28] interesting pod/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 16 05:32:55.937212 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:55.937113 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" podUID="28a4248f-5aab-4b95-9b69-16a227d57738" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 16 05:32:56.693634 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:56.693591 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:56.936043 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:56.936005 2561 patch_prober.go:28] interesting pod/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 16 05:32:56.936409 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:56.936057 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" podUID="28a4248f-5aab-4b95-9b69-16a227d57738" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 16 05:32:57.936289 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:57.936249 2561 patch_prober.go:28] interesting pod/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 16 05:32:57.936674 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:57.936306 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" podUID="28a4248f-5aab-4b95-9b69-16a227d57738" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 16 05:32:58.936547 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:58.936503 2561 patch_prober.go:28] interesting pod/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 16 05:32:58.936970 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:58.936575 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" podUID="28a4248f-5aab-4b95-9b69-16a227d57738" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 16 05:32:59.944935 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:59.944898 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:32:59.946071 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:32:59.946052 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:33:06.844794 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:06.844710 2561 generic.go:356] "Generic (PLEG): container finished" podID="23980ab8-c101-48e2-9a3f-d6e34bb85637" containerID="21e2505c25b8d81c5e7c4b74d56820593d184263b1eb26f86c3da171413d2ef7" exitCode=0 Jul 16 05:33:06.844794 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:06.844774 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" event={"ID":"23980ab8-c101-48e2-9a3f-d6e34bb85637","Type":"ContainerDied","Data":"21e2505c25b8d81c5e7c4b74d56820593d184263b1eb26f86c3da171413d2ef7"} Jul 16 05:33:06.845232 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:06.845168 2561 scope.go:117] "RemoveContainer" containerID="21e2505c25b8d81c5e7c4b74d56820593d184263b1eb26f86c3da171413d2ef7" Jul 16 05:33:06.846192 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:06.846163 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:33:06.846295 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:06.846237 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:33:06.846295 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:06.846282 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:33:07.850459 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:07.850419 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" event={"ID":"23980ab8-c101-48e2-9a3f-d6e34bb85637","Type":"ContainerStarted","Data":"a177c4b1d492e861dcb12d2a62d33ef4259bdc90b33c05fe79007f7fc6bbe74e"} Jul 16 05:33:08.421255 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:08.421216 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:33:08.421480 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:08.421454 2561 patch_prober.go:28] interesting pod/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 16 05:33:08.421567 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:08.421533 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" podUID="23980ab8-c101-48e2-9a3f-d6e34bb85637" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 16 05:33:09.422369 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:09.422327 2561 patch_prober.go:28] interesting pod/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 16 05:33:09.422797 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:09.422387 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" podUID="23980ab8-c101-48e2-9a3f-d6e34bb85637" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 16 05:33:10.422138 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:10.422096 2561 patch_prober.go:28] interesting pod/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 16 05:33:10.422313 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:10.422166 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" podUID="23980ab8-c101-48e2-9a3f-d6e34bb85637" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 16 05:33:11.422138 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:11.422104 2561 patch_prober.go:28] interesting pod/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 16 05:33:11.422554 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:11.422164 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" podUID="23980ab8-c101-48e2-9a3f-d6e34bb85637" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 16 05:33:12.425326 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:12.425292 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:33:12.425838 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:12.425715 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:33:12.426480 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:12.426461 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:33:28.444364 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:28.444324 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc"] Jul 16 05:33:28.444933 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:28.444588 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" podUID="28a4248f-5aab-4b95-9b69-16a227d57738" containerName="istio-proxy" containerID="cri-o://393fe4214e9c5a973e5b85d621fb5673885b2c3ea838e6babafa8ba2ba7431f5" gracePeriod=30 Jul 16 05:33:29.947357 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:29.947321 2561 patch_prober.go:28] interesting pod/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 05:33:29.947773 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:29.947377 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" podUID="28a4248f-5aab-4b95-9b69-16a227d57738" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 05:33:33.699485 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.699455 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:33:33.770426 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.770328 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-credential-socket\") pod \"28a4248f-5aab-4b95-9b69-16a227d57738\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " Jul 16 05:33:33.770426 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.770401 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mj4g\" (UniqueName: \"kubernetes.io/projected/28a4248f-5aab-4b95-9b69-16a227d57738-kube-api-access-4mj4g\") pod \"28a4248f-5aab-4b95-9b69-16a227d57738\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " Jul 16 05:33:33.770654 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.770440 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-workload-certs\") pod \"28a4248f-5aab-4b95-9b69-16a227d57738\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " Jul 16 05:33:33.770654 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.770468 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-workload-socket\") pod \"28a4248f-5aab-4b95-9b69-16a227d57738\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " Jul 16 05:33:33.770654 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.770509 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-istio-data\") pod \"28a4248f-5aab-4b95-9b69-16a227d57738\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " Jul 16 05:33:33.770654 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.770530 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28a4248f-5aab-4b95-9b69-16a227d57738-istiod-ca-cert\") pod \"28a4248f-5aab-4b95-9b69-16a227d57738\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " Jul 16 05:33:33.770654 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.770560 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28a4248f-5aab-4b95-9b69-16a227d57738-istio-podinfo\") pod \"28a4248f-5aab-4b95-9b69-16a227d57738\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " Jul 16 05:33:33.770654 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.770593 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28a4248f-5aab-4b95-9b69-16a227d57738-istio-token\") pod \"28a4248f-5aab-4b95-9b69-16a227d57738\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " Jul 16 05:33:33.770654 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.770628 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-istio-envoy\") pod \"28a4248f-5aab-4b95-9b69-16a227d57738\" (UID: \"28a4248f-5aab-4b95-9b69-16a227d57738\") " Jul 16 05:33:33.770654 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.770635 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "28a4248f-5aab-4b95-9b69-16a227d57738" (UID: "28a4248f-5aab-4b95-9b69-16a227d57738"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:33:33.771081 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.770869 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "28a4248f-5aab-4b95-9b69-16a227d57738" (UID: "28a4248f-5aab-4b95-9b69-16a227d57738"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:33:33.771081 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.770920 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "28a4248f-5aab-4b95-9b69-16a227d57738" (UID: "28a4248f-5aab-4b95-9b69-16a227d57738"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:33:33.771081 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.770951 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-workload-certs\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:33:33.771081 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.770972 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-credential-socket\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:33:33.771398 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.771097 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28a4248f-5aab-4b95-9b69-16a227d57738-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "28a4248f-5aab-4b95-9b69-16a227d57738" (UID: "28a4248f-5aab-4b95-9b69-16a227d57738"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 05:33:33.772024 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.771992 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-istio-data" (OuterVolumeSpecName: "istio-data") pod "28a4248f-5aab-4b95-9b69-16a227d57738" (UID: "28a4248f-5aab-4b95-9b69-16a227d57738"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:33:33.772979 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.772951 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28a4248f-5aab-4b95-9b69-16a227d57738-istio-token" (OuterVolumeSpecName: "istio-token") pod "28a4248f-5aab-4b95-9b69-16a227d57738" (UID: "28a4248f-5aab-4b95-9b69-16a227d57738"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:33:33.773070 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.772996 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28a4248f-5aab-4b95-9b69-16a227d57738-kube-api-access-4mj4g" (OuterVolumeSpecName: "kube-api-access-4mj4g") pod "28a4248f-5aab-4b95-9b69-16a227d57738" (UID: "28a4248f-5aab-4b95-9b69-16a227d57738"). InnerVolumeSpecName "kube-api-access-4mj4g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:33:33.773070 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.773043 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "28a4248f-5aab-4b95-9b69-16a227d57738" (UID: "28a4248f-5aab-4b95-9b69-16a227d57738"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:33:33.773366 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.773342 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/28a4248f-5aab-4b95-9b69-16a227d57738-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "28a4248f-5aab-4b95-9b69-16a227d57738" (UID: "28a4248f-5aab-4b95-9b69-16a227d57738"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 05:33:33.872020 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.871980 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28a4248f-5aab-4b95-9b69-16a227d57738-istio-token\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:33:33.872020 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.872013 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-istio-envoy\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:33:33.872020 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.872028 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mj4g\" (UniqueName: \"kubernetes.io/projected/28a4248f-5aab-4b95-9b69-16a227d57738-kube-api-access-4mj4g\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:33:33.872272 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.872042 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-workload-socket\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:33:33.872272 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.872053 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28a4248f-5aab-4b95-9b69-16a227d57738-istio-data\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:33:33.872272 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.872066 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28a4248f-5aab-4b95-9b69-16a227d57738-istiod-ca-cert\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:33:33.872272 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.872076 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28a4248f-5aab-4b95-9b69-16a227d57738-istio-podinfo\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:33:33.966801 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.966770 2561 generic.go:356] "Generic (PLEG): container finished" podID="28a4248f-5aab-4b95-9b69-16a227d57738" containerID="393fe4214e9c5a973e5b85d621fb5673885b2c3ea838e6babafa8ba2ba7431f5" exitCode=0 Jul 16 05:33:33.966970 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.966817 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" Jul 16 05:33:33.966970 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.966837 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" event={"ID":"28a4248f-5aab-4b95-9b69-16a227d57738","Type":"ContainerDied","Data":"393fe4214e9c5a973e5b85d621fb5673885b2c3ea838e6babafa8ba2ba7431f5"} Jul 16 05:33:33.966970 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.966881 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc" event={"ID":"28a4248f-5aab-4b95-9b69-16a227d57738","Type":"ContainerDied","Data":"b4eed80a337008fe6f615c83b173ed24225efc7e6ea0246aeffb98e6b2f0dfd6"} Jul 16 05:33:33.966970 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.966902 2561 scope.go:117] "RemoveContainer" containerID="393fe4214e9c5a973e5b85d621fb5673885b2c3ea838e6babafa8ba2ba7431f5" Jul 16 05:33:33.976544 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.976525 2561 scope.go:117] "RemoveContainer" containerID="53a12693a078da81ba2a4ad6f42c85611b411b2fa3e6bce8fff46245e0347c02" Jul 16 05:33:33.984093 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.984076 2561 scope.go:117] "RemoveContainer" containerID="393fe4214e9c5a973e5b85d621fb5673885b2c3ea838e6babafa8ba2ba7431f5" Jul 16 05:33:33.984343 ip-10-0-135-187 kubenswrapper[2561]: E0716 05:33:33.984326 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"393fe4214e9c5a973e5b85d621fb5673885b2c3ea838e6babafa8ba2ba7431f5\": container with ID starting with 393fe4214e9c5a973e5b85d621fb5673885b2c3ea838e6babafa8ba2ba7431f5 not found: ID does not exist" containerID="393fe4214e9c5a973e5b85d621fb5673885b2c3ea838e6babafa8ba2ba7431f5" Jul 16 05:33:33.984408 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.984352 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"393fe4214e9c5a973e5b85d621fb5673885b2c3ea838e6babafa8ba2ba7431f5"} err="failed to get container status \"393fe4214e9c5a973e5b85d621fb5673885b2c3ea838e6babafa8ba2ba7431f5\": rpc error: code = NotFound desc = could not find container \"393fe4214e9c5a973e5b85d621fb5673885b2c3ea838e6babafa8ba2ba7431f5\": container with ID starting with 393fe4214e9c5a973e5b85d621fb5673885b2c3ea838e6babafa8ba2ba7431f5 not found: ID does not exist" Jul 16 05:33:33.984408 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.984367 2561 scope.go:117] "RemoveContainer" containerID="53a12693a078da81ba2a4ad6f42c85611b411b2fa3e6bce8fff46245e0347c02" Jul 16 05:33:33.984600 ip-10-0-135-187 kubenswrapper[2561]: E0716 05:33:33.984583 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53a12693a078da81ba2a4ad6f42c85611b411b2fa3e6bce8fff46245e0347c02\": container with ID starting with 53a12693a078da81ba2a4ad6f42c85611b411b2fa3e6bce8fff46245e0347c02 not found: ID does not exist" containerID="53a12693a078da81ba2a4ad6f42c85611b411b2fa3e6bce8fff46245e0347c02" Jul 16 05:33:33.984644 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.984604 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53a12693a078da81ba2a4ad6f42c85611b411b2fa3e6bce8fff46245e0347c02"} err="failed to get container status \"53a12693a078da81ba2a4ad6f42c85611b411b2fa3e6bce8fff46245e0347c02\": rpc error: code = NotFound desc = could not find container \"53a12693a078da81ba2a4ad6f42c85611b411b2fa3e6bce8fff46245e0347c02\": container with ID starting with 53a12693a078da81ba2a4ad6f42c85611b411b2fa3e6bce8fff46245e0347c02 not found: ID does not exist" Jul 16 05:33:33.988557 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.988534 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc"] Jul 16 05:33:33.992499 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:33.992478 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ddd4fb-openshift-default-56cbc49c55-cwmqc"] Jul 16 05:33:34.100700 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:34.100657 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28a4248f-5aab-4b95-9b69-16a227d57738" path="/var/lib/kubelet/pods/28a4248f-5aab-4b95-9b69-16a227d57738/volumes" Jul 16 05:33:54.735303 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:54.735270 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb"] Jul 16 05:33:54.735820 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:54.735489 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" podUID="23980ab8-c101-48e2-9a3f-d6e34bb85637" containerName="istio-proxy" containerID="cri-o://a177c4b1d492e861dcb12d2a62d33ef4259bdc90b33c05fe79007f7fc6bbe74e" gracePeriod=30 Jul 16 05:33:55.774346 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.774316 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g"] Jul 16 05:33:55.774745 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.774622 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28a4248f-5aab-4b95-9b69-16a227d57738" containerName="istio-proxy" Jul 16 05:33:55.774745 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.774632 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="28a4248f-5aab-4b95-9b69-16a227d57738" containerName="istio-proxy" Jul 16 05:33:55.774745 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.774652 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28a4248f-5aab-4b95-9b69-16a227d57738" containerName="istio-proxy" Jul 16 05:33:55.774745 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.774658 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="28a4248f-5aab-4b95-9b69-16a227d57738" containerName="istio-proxy" Jul 16 05:33:55.774745 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.774730 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="28a4248f-5aab-4b95-9b69-16a227d57738" containerName="istio-proxy" Jul 16 05:33:55.774745 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.774739 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e937ad06-781d-4e47-85c1-0b50043c3ccf" containerName="istio-proxy" Jul 16 05:33:55.774745 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.774746 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="28a4248f-5aab-4b95-9b69-16a227d57738" containerName="istio-proxy" Jul 16 05:33:55.777614 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.777589 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.780124 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.780104 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-615edb-openshift-default-dockercfg-7dtms\"" Jul 16 05:33:55.792155 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.792126 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g"] Jul 16 05:33:55.866509 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.866470 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d328857d-cee4-42a1-bd00-638fb7139a0b-istiod-ca-cert\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.866509 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.866513 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-istio-envoy\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.866782 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.866536 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-workload-socket\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.866782 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.866570 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d328857d-cee4-42a1-bd00-638fb7139a0b-istio-podinfo\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.866782 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.866638 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d328857d-cee4-42a1-bd00-638fb7139a0b-istio-token\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.866782 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.866671 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-istio-data\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.866782 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.866727 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-credential-socket\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.866990 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.866794 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-workload-certs\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.866990 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.866894 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-727lx\" (UniqueName: \"kubernetes.io/projected/d328857d-cee4-42a1-bd00-638fb7139a0b-kube-api-access-727lx\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.967425 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.967387 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-credential-socket\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.967425 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.967427 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-workload-certs\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.967724 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.967490 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-727lx\" (UniqueName: \"kubernetes.io/projected/d328857d-cee4-42a1-bd00-638fb7139a0b-kube-api-access-727lx\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.967724 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.967526 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d328857d-cee4-42a1-bd00-638fb7139a0b-istiod-ca-cert\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.967724 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.967551 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-istio-envoy\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.967724 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.967581 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-workload-socket\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.967724 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.967605 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d328857d-cee4-42a1-bd00-638fb7139a0b-istio-podinfo\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.967724 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.967652 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d328857d-cee4-42a1-bd00-638fb7139a0b-istio-token\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.968036 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.967733 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-istio-data\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.968036 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.967861 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-credential-socket\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.968036 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.967918 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-workload-certs\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.968228 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.968084 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-workload-socket\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.968228 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.968199 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d328857d-cee4-42a1-bd00-638fb7139a0b-istiod-ca-cert\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.968392 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.968373 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-istio-data\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.970278 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.970254 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d328857d-cee4-42a1-bd00-638fb7139a0b-istio-podinfo\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.970372 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.970260 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-istio-envoy\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.976136 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.976107 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d328857d-cee4-42a1-bd00-638fb7139a0b-istio-token\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:55.976234 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:55.976217 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-727lx\" (UniqueName: \"kubernetes.io/projected/d328857d-cee4-42a1-bd00-638fb7139a0b-kube-api-access-727lx\") pod \"e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:56.089255 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:56.089224 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:56.226806 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:56.226776 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g"] Jul 16 05:33:56.228720 ip-10-0-135-187 kubenswrapper[2561]: W0716 05:33:56.228662 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd328857d_cee4_42a1_bd00_638fb7139a0b.slice/crio-14f0958e3b988cd5585a901a4279d0c6dfb08b6099d66de83832364eb9cc027b WatchSource:0}: Error finding container 14f0958e3b988cd5585a901a4279d0c6dfb08b6099d66de83832364eb9cc027b: Status 404 returned error can't find the container with id 14f0958e3b988cd5585a901a4279d0c6dfb08b6099d66de83832364eb9cc027b Jul 16 05:33:56.236545 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:56.236498 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:33:56.237050 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:56.237022 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:33:56.237148 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:56.237080 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:33:57.068749 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:57.068709 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" event={"ID":"d328857d-cee4-42a1-bd00-638fb7139a0b","Type":"ContainerStarted","Data":"7ad8945c3917bfd0bcc5642f628f463fe57385fe0b53c2681e74578d027bc165"} Jul 16 05:33:57.069259 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:57.068791 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" event={"ID":"d328857d-cee4-42a1-bd00-638fb7139a0b","Type":"ContainerStarted","Data":"14f0958e3b988cd5585a901a4279d0c6dfb08b6099d66de83832364eb9cc027b"} Jul 16 05:33:57.089503 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:57.089459 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:57.089765 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:57.089674 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" podStartSLOduration=2.089654108 podStartE2EDuration="2.089654108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:33:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:33:57.085981786 +0000 UTC m=+2627.458105129" watchObservedRunningTime="2026-07-16 05:33:57.089654108 +0000 UTC m=+2627.461777314" Jul 16 05:33:57.095006 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:57.094982 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:57.428619 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:57.428533 2561 patch_prober.go:28] interesting pod/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 05:33:57.428619 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:57.428593 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" podUID="23980ab8-c101-48e2-9a3f-d6e34bb85637" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 05:33:58.073701 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:58.073647 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:58.074789 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:58.074767 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:33:59.980806 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:33:59.980783 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:34:00.001861 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.001780 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-workload-socket\") pod \"23980ab8-c101-48e2-9a3f-d6e34bb85637\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " Jul 16 05:34:00.001995 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.001863 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23980ab8-c101-48e2-9a3f-d6e34bb85637-istio-podinfo\") pod \"23980ab8-c101-48e2-9a3f-d6e34bb85637\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " Jul 16 05:34:00.001995 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.001935 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvjww\" (UniqueName: \"kubernetes.io/projected/23980ab8-c101-48e2-9a3f-d6e34bb85637-kube-api-access-nvjww\") pod \"23980ab8-c101-48e2-9a3f-d6e34bb85637\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " Jul 16 05:34:00.001995 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.001979 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-workload-certs\") pod \"23980ab8-c101-48e2-9a3f-d6e34bb85637\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " Jul 16 05:34:00.002151 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.002003 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-credential-socket\") pod \"23980ab8-c101-48e2-9a3f-d6e34bb85637\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " Jul 16 05:34:00.002151 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.002031 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23980ab8-c101-48e2-9a3f-d6e34bb85637-istiod-ca-cert\") pod \"23980ab8-c101-48e2-9a3f-d6e34bb85637\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " Jul 16 05:34:00.002151 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.002073 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-istio-data\") pod \"23980ab8-c101-48e2-9a3f-d6e34bb85637\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " Jul 16 05:34:00.002151 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.002102 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-istio-envoy\") pod \"23980ab8-c101-48e2-9a3f-d6e34bb85637\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " Jul 16 05:34:00.002151 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.002133 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23980ab8-c101-48e2-9a3f-d6e34bb85637-istio-token\") pod \"23980ab8-c101-48e2-9a3f-d6e34bb85637\" (UID: \"23980ab8-c101-48e2-9a3f-d6e34bb85637\") " Jul 16 05:34:00.002375 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.002144 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "23980ab8-c101-48e2-9a3f-d6e34bb85637" (UID: "23980ab8-c101-48e2-9a3f-d6e34bb85637"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:34:00.002440 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.002376 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-workload-socket\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:34:00.003099 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.002654 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "23980ab8-c101-48e2-9a3f-d6e34bb85637" (UID: "23980ab8-c101-48e2-9a3f-d6e34bb85637"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:34:00.003099 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.003042 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "23980ab8-c101-48e2-9a3f-d6e34bb85637" (UID: "23980ab8-c101-48e2-9a3f-d6e34bb85637"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:34:00.003099 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.003043 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23980ab8-c101-48e2-9a3f-d6e34bb85637-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "23980ab8-c101-48e2-9a3f-d6e34bb85637" (UID: "23980ab8-c101-48e2-9a3f-d6e34bb85637"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 05:34:00.003864 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.003836 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-istio-data" (OuterVolumeSpecName: "istio-data") pod "23980ab8-c101-48e2-9a3f-d6e34bb85637" (UID: "23980ab8-c101-48e2-9a3f-d6e34bb85637"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:34:00.004862 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.004825 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23980ab8-c101-48e2-9a3f-d6e34bb85637-kube-api-access-nvjww" (OuterVolumeSpecName: "kube-api-access-nvjww") pod "23980ab8-c101-48e2-9a3f-d6e34bb85637" (UID: "23980ab8-c101-48e2-9a3f-d6e34bb85637"). InnerVolumeSpecName "kube-api-access-nvjww". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:34:00.004862 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.004838 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23980ab8-c101-48e2-9a3f-d6e34bb85637-istio-token" (OuterVolumeSpecName: "istio-token") pod "23980ab8-c101-48e2-9a3f-d6e34bb85637" (UID: "23980ab8-c101-48e2-9a3f-d6e34bb85637"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:34:00.005025 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.005004 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/23980ab8-c101-48e2-9a3f-d6e34bb85637-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "23980ab8-c101-48e2-9a3f-d6e34bb85637" (UID: "23980ab8-c101-48e2-9a3f-d6e34bb85637"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 05:34:00.005731 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.005703 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "23980ab8-c101-48e2-9a3f-d6e34bb85637" (UID: "23980ab8-c101-48e2-9a3f-d6e34bb85637"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:34:00.084851 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.084811 2561 generic.go:356] "Generic (PLEG): container finished" podID="23980ab8-c101-48e2-9a3f-d6e34bb85637" containerID="a177c4b1d492e861dcb12d2a62d33ef4259bdc90b33c05fe79007f7fc6bbe74e" exitCode=0 Jul 16 05:34:00.084851 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.084881 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" Jul 16 05:34:00.084851 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.084891 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" event={"ID":"23980ab8-c101-48e2-9a3f-d6e34bb85637","Type":"ContainerDied","Data":"a177c4b1d492e861dcb12d2a62d33ef4259bdc90b33c05fe79007f7fc6bbe74e"} Jul 16 05:34:00.084851 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.084954 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb" event={"ID":"23980ab8-c101-48e2-9a3f-d6e34bb85637","Type":"ContainerDied","Data":"7c7c2b22db892428cca5c41ca218fa58c4c3ee761fc0663a6ed9a48aaa2bf5d6"} Jul 16 05:34:00.084851 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.084985 2561 scope.go:117] "RemoveContainer" containerID="a177c4b1d492e861dcb12d2a62d33ef4259bdc90b33c05fe79007f7fc6bbe74e" Jul 16 05:34:00.097014 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.096829 2561 scope.go:117] "RemoveContainer" containerID="21e2505c25b8d81c5e7c4b74d56820593d184263b1eb26f86c3da171413d2ef7" Jul 16 05:34:00.106317 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.103425 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23980ab8-c101-48e2-9a3f-d6e34bb85637-istio-podinfo\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:34:00.106317 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.103461 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvjww\" (UniqueName: \"kubernetes.io/projected/23980ab8-c101-48e2-9a3f-d6e34bb85637-kube-api-access-nvjww\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:34:00.106317 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.103477 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-workload-certs\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:34:00.106317 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.103492 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-credential-socket\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:34:00.106317 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.103506 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23980ab8-c101-48e2-9a3f-d6e34bb85637-istiod-ca-cert\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:34:00.106317 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.103525 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-istio-data\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:34:00.106317 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.103538 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23980ab8-c101-48e2-9a3f-d6e34bb85637-istio-envoy\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:34:00.106317 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.103550 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23980ab8-c101-48e2-9a3f-d6e34bb85637-istio-token\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:34:00.110249 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.110230 2561 scope.go:117] "RemoveContainer" containerID="a177c4b1d492e861dcb12d2a62d33ef4259bdc90b33c05fe79007f7fc6bbe74e" Jul 16 05:34:00.110759 ip-10-0-135-187 kubenswrapper[2561]: E0716 05:34:00.110603 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a177c4b1d492e861dcb12d2a62d33ef4259bdc90b33c05fe79007f7fc6bbe74e\": container with ID starting with a177c4b1d492e861dcb12d2a62d33ef4259bdc90b33c05fe79007f7fc6bbe74e not found: ID does not exist" containerID="a177c4b1d492e861dcb12d2a62d33ef4259bdc90b33c05fe79007f7fc6bbe74e" Jul 16 05:34:00.110759 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.110638 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a177c4b1d492e861dcb12d2a62d33ef4259bdc90b33c05fe79007f7fc6bbe74e"} err="failed to get container status \"a177c4b1d492e861dcb12d2a62d33ef4259bdc90b33c05fe79007f7fc6bbe74e\": rpc error: code = NotFound desc = could not find container \"a177c4b1d492e861dcb12d2a62d33ef4259bdc90b33c05fe79007f7fc6bbe74e\": container with ID starting with a177c4b1d492e861dcb12d2a62d33ef4259bdc90b33c05fe79007f7fc6bbe74e not found: ID does not exist" Jul 16 05:34:00.110759 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.110665 2561 scope.go:117] "RemoveContainer" containerID="21e2505c25b8d81c5e7c4b74d56820593d184263b1eb26f86c3da171413d2ef7" Jul 16 05:34:00.111246 ip-10-0-135-187 kubenswrapper[2561]: E0716 05:34:00.111198 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21e2505c25b8d81c5e7c4b74d56820593d184263b1eb26f86c3da171413d2ef7\": container with ID starting with 21e2505c25b8d81c5e7c4b74d56820593d184263b1eb26f86c3da171413d2ef7 not found: ID does not exist" containerID="21e2505c25b8d81c5e7c4b74d56820593d184263b1eb26f86c3da171413d2ef7" Jul 16 05:34:00.111314 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.111256 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21e2505c25b8d81c5e7c4b74d56820593d184263b1eb26f86c3da171413d2ef7"} err="failed to get container status \"21e2505c25b8d81c5e7c4b74d56820593d184263b1eb26f86c3da171413d2ef7\": rpc error: code = NotFound desc = could not find container \"21e2505c25b8d81c5e7c4b74d56820593d184263b1eb26f86c3da171413d2ef7\": container with ID starting with 21e2505c25b8d81c5e7c4b74d56820593d184263b1eb26f86c3da171413d2ef7 not found: ID does not exist" Jul 16 05:34:00.112444 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.112425 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb"] Jul 16 05:34:00.116530 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:00.116511 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-2797de-openshift-default-6cd45565fc-x88kb"] Jul 16 05:34:02.099827 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:02.099792 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23980ab8-c101-48e2-9a3f-d6e34bb85637" path="/var/lib/kubelet/pods/23980ab8-c101-48e2-9a3f-d6e34bb85637/volumes" Jul 16 05:34:04.378618 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.378583 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856"] Jul 16 05:34:04.379100 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.379079 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23980ab8-c101-48e2-9a3f-d6e34bb85637" containerName="istio-proxy" Jul 16 05:34:04.379161 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.379104 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="23980ab8-c101-48e2-9a3f-d6e34bb85637" containerName="istio-proxy" Jul 16 05:34:04.379161 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.379131 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23980ab8-c101-48e2-9a3f-d6e34bb85637" containerName="istio-proxy" Jul 16 05:34:04.379161 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.379140 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="23980ab8-c101-48e2-9a3f-d6e34bb85637" containerName="istio-proxy" Jul 16 05:34:04.379279 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.379212 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="23980ab8-c101-48e2-9a3f-d6e34bb85637" containerName="istio-proxy" Jul 16 05:34:04.379279 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.379225 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="23980ab8-c101-48e2-9a3f-d6e34bb85637" containerName="istio-proxy" Jul 16 05:34:04.384002 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.383976 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.386342 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.386311 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-e36375-openshift-default-dockercfg-xf6gn\"" Jul 16 05:34:04.393522 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.393495 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856"] Jul 16 05:34:04.442251 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.442193 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-istio-envoy\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.442466 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.442259 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-istio-data\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.442466 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.442291 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3f65dab-2090-480d-b77b-64e7977d851e-istio-podinfo\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.442466 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.442335 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6nsq\" (UniqueName: \"kubernetes.io/projected/d3f65dab-2090-480d-b77b-64e7977d851e-kube-api-access-n6nsq\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.442466 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.442440 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-credential-socket\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.442750 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.442483 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3f65dab-2090-480d-b77b-64e7977d851e-istiod-ca-cert\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.442750 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.442529 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-workload-socket\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.442750 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.442606 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3f65dab-2090-480d-b77b-64e7977d851e-istio-token\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.442750 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.442668 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-workload-certs\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.543946 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.543906 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-credential-socket\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.544152 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.543952 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3f65dab-2090-480d-b77b-64e7977d851e-istiod-ca-cert\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.544152 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.543991 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-workload-socket\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.544152 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.544026 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3f65dab-2090-480d-b77b-64e7977d851e-istio-token\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.544152 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.544078 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-workload-certs\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.544152 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.544125 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-istio-envoy\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.544152 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.544147 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-istio-data\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.544473 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.544173 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3f65dab-2090-480d-b77b-64e7977d851e-istio-podinfo\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.544473 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.544206 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6nsq\" (UniqueName: \"kubernetes.io/projected/d3f65dab-2090-480d-b77b-64e7977d851e-kube-api-access-n6nsq\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.544473 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.544407 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-credential-socket\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.544636 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.544564 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-workload-socket\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.544731 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.544651 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-workload-certs\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.544731 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.544657 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-istio-data\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.544851 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.544830 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3f65dab-2090-480d-b77b-64e7977d851e-istiod-ca-cert\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.546733 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.546714 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-istio-envoy\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.546873 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.546856 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3f65dab-2090-480d-b77b-64e7977d851e-istio-podinfo\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.551745 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.551719 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3f65dab-2090-480d-b77b-64e7977d851e-istio-token\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.552316 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.552291 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6nsq\" (UniqueName: \"kubernetes.io/projected/d3f65dab-2090-480d-b77b-64e7977d851e-kube-api-access-n6nsq\") pod \"e2e-shared-b-e36375-openshift-default-58cb665687-7z856\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.698457 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.698355 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:04.861355 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.861263 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856"] Jul 16 05:34:04.863710 ip-10-0-135-187 kubenswrapper[2561]: W0716 05:34:04.863661 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3f65dab_2090_480d_b77b_64e7977d851e.slice/crio-c6cab8becfe203af3564ffbcbcd6dcb357619ec6eb3578483081dd681eb217f3 WatchSource:0}: Error finding container c6cab8becfe203af3564ffbcbcd6dcb357619ec6eb3578483081dd681eb217f3: Status 404 returned error can't find the container with id c6cab8becfe203af3564ffbcbcd6dcb357619ec6eb3578483081dd681eb217f3 Jul 16 05:34:04.866751 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.866039 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:34:04.866751 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.866125 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:34:04.866751 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:04.866168 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:34:05.110198 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:05.110160 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" event={"ID":"d3f65dab-2090-480d-b77b-64e7977d851e","Type":"ContainerStarted","Data":"f86b2cac58fa5574885efc4ae3bb9e8f5c5d274de7a3ef6254fb47a2f37e06bf"} Jul 16 05:34:05.110376 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:05.110221 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" event={"ID":"d3f65dab-2090-480d-b77b-64e7977d851e","Type":"ContainerStarted","Data":"c6cab8becfe203af3564ffbcbcd6dcb357619ec6eb3578483081dd681eb217f3"} Jul 16 05:34:05.128073 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:05.128027 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" podStartSLOduration=1.1280116119999999 podStartE2EDuration="1.128011612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:34:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:34:05.127299168 +0000 UTC m=+2635.499422394" watchObservedRunningTime="2026-07-16 05:34:05.128011612 +0000 UTC m=+2635.500134875" Jul 16 05:34:05.699349 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:05.699307 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:05.703846 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:05.703817 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:06.120198 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:06.120163 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:06.121260 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:06.121240 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:12.150703 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:12.150641 2561 generic.go:356] "Generic (PLEG): container finished" podID="d328857d-cee4-42a1-bd00-638fb7139a0b" containerID="7ad8945c3917bfd0bcc5642f628f463fe57385fe0b53c2681e74578d027bc165" exitCode=0 Jul 16 05:34:12.151123 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:12.150719 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" event={"ID":"d328857d-cee4-42a1-bd00-638fb7139a0b","Type":"ContainerDied","Data":"7ad8945c3917bfd0bcc5642f628f463fe57385fe0b53c2681e74578d027bc165"} Jul 16 05:34:12.151183 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:12.151162 2561 scope.go:117] "RemoveContainer" containerID="7ad8945c3917bfd0bcc5642f628f463fe57385fe0b53c2681e74578d027bc165" Jul 16 05:34:12.152155 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:12.152130 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:34:12.152211 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:12.152202 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:34:12.152255 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:12.152240 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:34:13.074143 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:13.074100 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:34:13.157046 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:13.157010 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" event={"ID":"d328857d-cee4-42a1-bd00-638fb7139a0b","Type":"ContainerStarted","Data":"64e75d197842f9384745cf102388683f08f843d38655d02a1a2b9d583c1f94ae"} Jul 16 05:34:14.091616 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:14.091574 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:34:14.091845 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:14.091756 2561 patch_prober.go:28] interesting pod/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 16 05:34:14.091845 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:14.091802 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" podUID="d328857d-cee4-42a1-bd00-638fb7139a0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 16 05:34:15.090178 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:15.090130 2561 patch_prober.go:28] interesting pod/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 16 05:34:15.090724 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:15.090207 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" podUID="d328857d-cee4-42a1-bd00-638fb7139a0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 16 05:34:16.089969 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:16.089917 2561 patch_prober.go:28] interesting pod/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 16 05:34:16.090247 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:16.090204 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" podUID="d328857d-cee4-42a1-bd00-638fb7139a0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 16 05:34:17.090728 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:17.090325 2561 patch_prober.go:28] interesting pod/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 16 05:34:17.090728 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:17.090392 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" podUID="d328857d-cee4-42a1-bd00-638fb7139a0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 16 05:34:18.094248 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:18.094215 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:34:18.102017 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:18.101989 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:34:18.102160 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:18.102084 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:34:19.194047 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:19.194009 2561 generic.go:356] "Generic (PLEG): container finished" podID="d3f65dab-2090-480d-b77b-64e7977d851e" containerID="f86b2cac58fa5574885efc4ae3bb9e8f5c5d274de7a3ef6254fb47a2f37e06bf" exitCode=0 Jul 16 05:34:19.194430 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:19.194064 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" event={"ID":"d3f65dab-2090-480d-b77b-64e7977d851e","Type":"ContainerDied","Data":"f86b2cac58fa5574885efc4ae3bb9e8f5c5d274de7a3ef6254fb47a2f37e06bf"} Jul 16 05:34:19.194485 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:19.194454 2561 scope.go:117] "RemoveContainer" containerID="f86b2cac58fa5574885efc4ae3bb9e8f5c5d274de7a3ef6254fb47a2f37e06bf" Jul 16 05:34:19.195471 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:19.195441 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:34:19.195582 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:19.195509 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:34:19.195582 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:19.195540 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 05:34:20.201542 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:20.201498 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" event={"ID":"d3f65dab-2090-480d-b77b-64e7977d851e","Type":"ContainerStarted","Data":"5eff9ffa26cfc63c810a1d3234630391342541b42c868f0b5b885ad6d396b89e"} Jul 16 05:34:20.698575 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:20.698539 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:20.699863 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:20.699837 2561 patch_prober.go:28] interesting pod/e2e-shared-b-e36375-openshift-default-58cb665687-7z856 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 16 05:34:20.700004 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:20.699902 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" podUID="d3f65dab-2090-480d-b77b-64e7977d851e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 16 05:34:21.120460 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:21.120422 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:21.699248 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:21.699209 2561 patch_prober.go:28] interesting pod/e2e-shared-b-e36375-openshift-default-58cb665687-7z856 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 16 05:34:21.699763 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:21.699279 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" podUID="d3f65dab-2090-480d-b77b-64e7977d851e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 16 05:34:22.699304 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:22.699271 2561 patch_prober.go:28] interesting pod/e2e-shared-b-e36375-openshift-default-58cb665687-7z856 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 16 05:34:22.699739 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:22.699330 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" podUID="d3f65dab-2090-480d-b77b-64e7977d851e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 16 05:34:23.698984 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:23.698936 2561 patch_prober.go:28] interesting pod/e2e-shared-b-e36375-openshift-default-58cb665687-7z856 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 16 05:34:23.699169 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:23.699020 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" podUID="d3f65dab-2090-480d-b77b-64e7977d851e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 16 05:34:24.703068 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:24.703031 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:34:24.703890 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:34:24.703870 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:40:28.848063 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:28.848033 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g"] Jul 16 05:40:28.848521 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:28.848297 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" podUID="d328857d-cee4-42a1-bd00-638fb7139a0b" containerName="istio-proxy" containerID="cri-o://64e75d197842f9384745cf102388683f08f843d38655d02a1a2b9d583c1f94ae" gracePeriod=30 Jul 16 05:40:33.096810 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:33.096771 2561 patch_prober.go:28] interesting pod/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 05:40:33.097298 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:33.096828 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" podUID="d328857d-cee4-42a1-bd00-638fb7139a0b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 05:40:34.090669 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.090642 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:40:34.268302 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.268214 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d328857d-cee4-42a1-bd00-638fb7139a0b-istiod-ca-cert\") pod \"d328857d-cee4-42a1-bd00-638fb7139a0b\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " Jul 16 05:40:34.268302 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.268255 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-727lx\" (UniqueName: \"kubernetes.io/projected/d328857d-cee4-42a1-bd00-638fb7139a0b-kube-api-access-727lx\") pod \"d328857d-cee4-42a1-bd00-638fb7139a0b\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " Jul 16 05:40:34.268302 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.268291 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-istio-envoy\") pod \"d328857d-cee4-42a1-bd00-638fb7139a0b\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " Jul 16 05:40:34.268854 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.268317 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d328857d-cee4-42a1-bd00-638fb7139a0b-istio-podinfo\") pod \"d328857d-cee4-42a1-bd00-638fb7139a0b\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " Jul 16 05:40:34.268854 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.268361 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d328857d-cee4-42a1-bd00-638fb7139a0b-istio-token\") pod \"d328857d-cee4-42a1-bd00-638fb7139a0b\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " Jul 16 05:40:34.268854 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.268388 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-credential-socket\") pod \"d328857d-cee4-42a1-bd00-638fb7139a0b\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " Jul 16 05:40:34.268854 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.268551 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-workload-socket\") pod \"d328857d-cee4-42a1-bd00-638fb7139a0b\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " Jul 16 05:40:34.268854 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.268677 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-istio-data\") pod \"d328857d-cee4-42a1-bd00-638fb7139a0b\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " Jul 16 05:40:34.268854 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.268724 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d328857d-cee4-42a1-bd00-638fb7139a0b" (UID: "d328857d-cee4-42a1-bd00-638fb7139a0b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:40:34.268854 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.268731 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d328857d-cee4-42a1-bd00-638fb7139a0b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d328857d-cee4-42a1-bd00-638fb7139a0b" (UID: "d328857d-cee4-42a1-bd00-638fb7139a0b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 05:40:34.268854 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.268754 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-workload-certs\") pod \"d328857d-cee4-42a1-bd00-638fb7139a0b\" (UID: \"d328857d-cee4-42a1-bd00-638fb7139a0b\") " Jul 16 05:40:34.269209 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.268929 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d328857d-cee4-42a1-bd00-638fb7139a0b" (UID: "d328857d-cee4-42a1-bd00-638fb7139a0b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:40:34.269209 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.269001 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d328857d-cee4-42a1-bd00-638fb7139a0b" (UID: "d328857d-cee4-42a1-bd00-638fb7139a0b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:40:34.269209 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.269084 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-workload-certs\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:40:34.269209 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.269107 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d328857d-cee4-42a1-bd00-638fb7139a0b-istiod-ca-cert\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:40:34.269209 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.269120 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-credential-socket\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:40:34.269209 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.269135 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-workload-socket\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:40:34.269727 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.269704 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-istio-data" (OuterVolumeSpecName: "istio-data") pod "d328857d-cee4-42a1-bd00-638fb7139a0b" (UID: "d328857d-cee4-42a1-bd00-638fb7139a0b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:40:34.270789 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.270764 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d328857d-cee4-42a1-bd00-638fb7139a0b-istio-token" (OuterVolumeSpecName: "istio-token") pod "d328857d-cee4-42a1-bd00-638fb7139a0b" (UID: "d328857d-cee4-42a1-bd00-638fb7139a0b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:40:34.270977 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.270954 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d328857d-cee4-42a1-bd00-638fb7139a0b-kube-api-access-727lx" (OuterVolumeSpecName: "kube-api-access-727lx") pod "d328857d-cee4-42a1-bd00-638fb7139a0b" (UID: "d328857d-cee4-42a1-bd00-638fb7139a0b"). InnerVolumeSpecName "kube-api-access-727lx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:40:34.271176 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.271159 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d328857d-cee4-42a1-bd00-638fb7139a0b" (UID: "d328857d-cee4-42a1-bd00-638fb7139a0b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:40:34.271216 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.271168 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d328857d-cee4-42a1-bd00-638fb7139a0b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d328857d-cee4-42a1-bd00-638fb7139a0b" (UID: "d328857d-cee4-42a1-bd00-638fb7139a0b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 05:40:34.370092 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.370048 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-727lx\" (UniqueName: \"kubernetes.io/projected/d328857d-cee4-42a1-bd00-638fb7139a0b-kube-api-access-727lx\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:40:34.370092 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.370084 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-istio-envoy\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:40:34.370092 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.370094 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d328857d-cee4-42a1-bd00-638fb7139a0b-istio-podinfo\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:40:34.370092 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.370101 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d328857d-cee4-42a1-bd00-638fb7139a0b-istio-token\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:40:34.370372 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.370110 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d328857d-cee4-42a1-bd00-638fb7139a0b-istio-data\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:40:34.864473 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.864438 2561 generic.go:356] "Generic (PLEG): container finished" podID="d328857d-cee4-42a1-bd00-638fb7139a0b" containerID="64e75d197842f9384745cf102388683f08f843d38655d02a1a2b9d583c1f94ae" exitCode=0 Jul 16 05:40:34.864666 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.864488 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" Jul 16 05:40:34.864666 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.864526 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" event={"ID":"d328857d-cee4-42a1-bd00-638fb7139a0b","Type":"ContainerDied","Data":"64e75d197842f9384745cf102388683f08f843d38655d02a1a2b9d583c1f94ae"} Jul 16 05:40:34.864666 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.864585 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g" event={"ID":"d328857d-cee4-42a1-bd00-638fb7139a0b","Type":"ContainerDied","Data":"14f0958e3b988cd5585a901a4279d0c6dfb08b6099d66de83832364eb9cc027b"} Jul 16 05:40:34.864666 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.864614 2561 scope.go:117] "RemoveContainer" containerID="64e75d197842f9384745cf102388683f08f843d38655d02a1a2b9d583c1f94ae" Jul 16 05:40:34.873584 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.873567 2561 scope.go:117] "RemoveContainer" containerID="7ad8945c3917bfd0bcc5642f628f463fe57385fe0b53c2681e74578d027bc165" Jul 16 05:40:34.881260 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.881243 2561 scope.go:117] "RemoveContainer" containerID="64e75d197842f9384745cf102388683f08f843d38655d02a1a2b9d583c1f94ae" Jul 16 05:40:34.881514 ip-10-0-135-187 kubenswrapper[2561]: E0716 05:40:34.881494 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64e75d197842f9384745cf102388683f08f843d38655d02a1a2b9d583c1f94ae\": container with ID starting with 64e75d197842f9384745cf102388683f08f843d38655d02a1a2b9d583c1f94ae not found: ID does not exist" containerID="64e75d197842f9384745cf102388683f08f843d38655d02a1a2b9d583c1f94ae" Jul 16 05:40:34.881571 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.881523 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64e75d197842f9384745cf102388683f08f843d38655d02a1a2b9d583c1f94ae"} err="failed to get container status \"64e75d197842f9384745cf102388683f08f843d38655d02a1a2b9d583c1f94ae\": rpc error: code = NotFound desc = could not find container \"64e75d197842f9384745cf102388683f08f843d38655d02a1a2b9d583c1f94ae\": container with ID starting with 64e75d197842f9384745cf102388683f08f843d38655d02a1a2b9d583c1f94ae not found: ID does not exist" Jul 16 05:40:34.881571 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.881541 2561 scope.go:117] "RemoveContainer" containerID="7ad8945c3917bfd0bcc5642f628f463fe57385fe0b53c2681e74578d027bc165" Jul 16 05:40:34.881830 ip-10-0-135-187 kubenswrapper[2561]: E0716 05:40:34.881812 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ad8945c3917bfd0bcc5642f628f463fe57385fe0b53c2681e74578d027bc165\": container with ID starting with 7ad8945c3917bfd0bcc5642f628f463fe57385fe0b53c2681e74578d027bc165 not found: ID does not exist" containerID="7ad8945c3917bfd0bcc5642f628f463fe57385fe0b53c2681e74578d027bc165" Jul 16 05:40:34.881894 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.881835 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ad8945c3917bfd0bcc5642f628f463fe57385fe0b53c2681e74578d027bc165"} err="failed to get container status \"7ad8945c3917bfd0bcc5642f628f463fe57385fe0b53c2681e74578d027bc165\": rpc error: code = NotFound desc = could not find container \"7ad8945c3917bfd0bcc5642f628f463fe57385fe0b53c2681e74578d027bc165\": container with ID starting with 7ad8945c3917bfd0bcc5642f628f463fe57385fe0b53c2681e74578d027bc165 not found: ID does not exist" Jul 16 05:40:34.885071 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.885048 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g"] Jul 16 05:40:34.888928 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:34.888907 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-615edb-openshift-default-b64bf8475-srx9g"] Jul 16 05:40:36.100600 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:36.100564 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d328857d-cee4-42a1-bd00-638fb7139a0b" path="/var/lib/kubelet/pods/d328857d-cee4-42a1-bd00-638fb7139a0b/volumes" Jul 16 05:40:54.992706 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:54.992655 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856"] Jul 16 05:40:54.993159 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:40:54.992957 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" podUID="d3f65dab-2090-480d-b77b-64e7977d851e" containerName="istio-proxy" containerID="cri-o://5eff9ffa26cfc63c810a1d3234630391342541b42c868f0b5b885ad6d396b89e" gracePeriod=30 Jul 16 05:41:00.247491 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.247436 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:41:00.292707 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.292650 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3f65dab-2090-480d-b77b-64e7977d851e-istio-token\") pod \"d3f65dab-2090-480d-b77b-64e7977d851e\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " Jul 16 05:41:00.292886 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.292803 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-workload-certs\") pod \"d3f65dab-2090-480d-b77b-64e7977d851e\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " Jul 16 05:41:00.292886 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.292839 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3f65dab-2090-480d-b77b-64e7977d851e-istiod-ca-cert\") pod \"d3f65dab-2090-480d-b77b-64e7977d851e\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " Jul 16 05:41:00.292886 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.292862 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-workload-socket\") pod \"d3f65dab-2090-480d-b77b-64e7977d851e\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " Jul 16 05:41:00.293052 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.292888 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-istio-envoy\") pod \"d3f65dab-2090-480d-b77b-64e7977d851e\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " Jul 16 05:41:00.293052 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.292915 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-credential-socket\") pod \"d3f65dab-2090-480d-b77b-64e7977d851e\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " Jul 16 05:41:00.293052 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.292963 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-istio-data\") pod \"d3f65dab-2090-480d-b77b-64e7977d851e\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " Jul 16 05:41:00.293052 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.292989 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3f65dab-2090-480d-b77b-64e7977d851e-istio-podinfo\") pod \"d3f65dab-2090-480d-b77b-64e7977d851e\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " Jul 16 05:41:00.293052 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.293015 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6nsq\" (UniqueName: \"kubernetes.io/projected/d3f65dab-2090-480d-b77b-64e7977d851e-kube-api-access-n6nsq\") pod \"d3f65dab-2090-480d-b77b-64e7977d851e\" (UID: \"d3f65dab-2090-480d-b77b-64e7977d851e\") " Jul 16 05:41:00.293303 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.293133 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d3f65dab-2090-480d-b77b-64e7977d851e" (UID: "d3f65dab-2090-480d-b77b-64e7977d851e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:41:00.293303 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.293150 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d3f65dab-2090-480d-b77b-64e7977d851e" (UID: "d3f65dab-2090-480d-b77b-64e7977d851e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:41:00.293303 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.293263 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d3f65dab-2090-480d-b77b-64e7977d851e" (UID: "d3f65dab-2090-480d-b77b-64e7977d851e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:41:00.293458 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.293306 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3f65dab-2090-480d-b77b-64e7977d851e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d3f65dab-2090-480d-b77b-64e7977d851e" (UID: "d3f65dab-2090-480d-b77b-64e7977d851e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 05:41:00.293458 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.293366 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-workload-certs\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:41:00.293458 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.293385 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-workload-socket\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:41:00.293458 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.293399 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-credential-socket\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:41:00.293843 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.293816 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-istio-data" (OuterVolumeSpecName: "istio-data") pod "d3f65dab-2090-480d-b77b-64e7977d851e" (UID: "d3f65dab-2090-480d-b77b-64e7977d851e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:41:00.295099 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.295066 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3f65dab-2090-480d-b77b-64e7977d851e-istio-token" (OuterVolumeSpecName: "istio-token") pod "d3f65dab-2090-480d-b77b-64e7977d851e" (UID: "d3f65dab-2090-480d-b77b-64e7977d851e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:41:00.295459 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.295436 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d3f65dab-2090-480d-b77b-64e7977d851e" (UID: "d3f65dab-2090-480d-b77b-64e7977d851e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:41:00.295742 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.295720 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3f65dab-2090-480d-b77b-64e7977d851e-kube-api-access-n6nsq" (OuterVolumeSpecName: "kube-api-access-n6nsq") pod "d3f65dab-2090-480d-b77b-64e7977d851e" (UID: "d3f65dab-2090-480d-b77b-64e7977d851e"). InnerVolumeSpecName "kube-api-access-n6nsq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:41:00.295819 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.295748 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d3f65dab-2090-480d-b77b-64e7977d851e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d3f65dab-2090-480d-b77b-64e7977d851e" (UID: "d3f65dab-2090-480d-b77b-64e7977d851e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 05:41:00.394034 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.393996 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-istio-data\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:41:00.394034 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.394028 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3f65dab-2090-480d-b77b-64e7977d851e-istio-podinfo\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:41:00.394217 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.394044 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6nsq\" (UniqueName: \"kubernetes.io/projected/d3f65dab-2090-480d-b77b-64e7977d851e-kube-api-access-n6nsq\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:41:00.394217 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.394055 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3f65dab-2090-480d-b77b-64e7977d851e-istio-token\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:41:00.394217 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.394066 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3f65dab-2090-480d-b77b-64e7977d851e-istiod-ca-cert\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:41:00.394217 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.394076 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3f65dab-2090-480d-b77b-64e7977d851e-istio-envoy\") on node \"ip-10-0-135-187.ec2.internal\" DevicePath \"\"" Jul 16 05:41:00.980972 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.980934 2561 generic.go:356] "Generic (PLEG): container finished" podID="d3f65dab-2090-480d-b77b-64e7977d851e" containerID="5eff9ffa26cfc63c810a1d3234630391342541b42c868f0b5b885ad6d396b89e" exitCode=0 Jul 16 05:41:00.981186 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.980988 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" Jul 16 05:41:00.981186 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.981020 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" event={"ID":"d3f65dab-2090-480d-b77b-64e7977d851e","Type":"ContainerDied","Data":"5eff9ffa26cfc63c810a1d3234630391342541b42c868f0b5b885ad6d396b89e"} Jul 16 05:41:00.981186 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.981088 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856" event={"ID":"d3f65dab-2090-480d-b77b-64e7977d851e","Type":"ContainerDied","Data":"c6cab8becfe203af3564ffbcbcd6dcb357619ec6eb3578483081dd681eb217f3"} Jul 16 05:41:00.981186 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.981128 2561 scope.go:117] "RemoveContainer" containerID="5eff9ffa26cfc63c810a1d3234630391342541b42c868f0b5b885ad6d396b89e" Jul 16 05:41:00.990383 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.990368 2561 scope.go:117] "RemoveContainer" containerID="f86b2cac58fa5574885efc4ae3bb9e8f5c5d274de7a3ef6254fb47a2f37e06bf" Jul 16 05:41:00.998088 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.998058 2561 scope.go:117] "RemoveContainer" containerID="5eff9ffa26cfc63c810a1d3234630391342541b42c868f0b5b885ad6d396b89e" Jul 16 05:41:00.998321 ip-10-0-135-187 kubenswrapper[2561]: E0716 05:41:00.998303 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5eff9ffa26cfc63c810a1d3234630391342541b42c868f0b5b885ad6d396b89e\": container with ID starting with 5eff9ffa26cfc63c810a1d3234630391342541b42c868f0b5b885ad6d396b89e not found: ID does not exist" containerID="5eff9ffa26cfc63c810a1d3234630391342541b42c868f0b5b885ad6d396b89e" Jul 16 05:41:00.998367 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.998339 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eff9ffa26cfc63c810a1d3234630391342541b42c868f0b5b885ad6d396b89e"} err="failed to get container status \"5eff9ffa26cfc63c810a1d3234630391342541b42c868f0b5b885ad6d396b89e\": rpc error: code = NotFound desc = could not find container \"5eff9ffa26cfc63c810a1d3234630391342541b42c868f0b5b885ad6d396b89e\": container with ID starting with 5eff9ffa26cfc63c810a1d3234630391342541b42c868f0b5b885ad6d396b89e not found: ID does not exist" Jul 16 05:41:00.998367 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.998357 2561 scope.go:117] "RemoveContainer" containerID="f86b2cac58fa5574885efc4ae3bb9e8f5c5d274de7a3ef6254fb47a2f37e06bf" Jul 16 05:41:00.998652 ip-10-0-135-187 kubenswrapper[2561]: E0716 05:41:00.998601 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f86b2cac58fa5574885efc4ae3bb9e8f5c5d274de7a3ef6254fb47a2f37e06bf\": container with ID starting with f86b2cac58fa5574885efc4ae3bb9e8f5c5d274de7a3ef6254fb47a2f37e06bf not found: ID does not exist" containerID="f86b2cac58fa5574885efc4ae3bb9e8f5c5d274de7a3ef6254fb47a2f37e06bf" Jul 16 05:41:00.998771 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:00.998646 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f86b2cac58fa5574885efc4ae3bb9e8f5c5d274de7a3ef6254fb47a2f37e06bf"} err="failed to get container status \"f86b2cac58fa5574885efc4ae3bb9e8f5c5d274de7a3ef6254fb47a2f37e06bf\": rpc error: code = NotFound desc = could not find container \"f86b2cac58fa5574885efc4ae3bb9e8f5c5d274de7a3ef6254fb47a2f37e06bf\": container with ID starting with f86b2cac58fa5574885efc4ae3bb9e8f5c5d274de7a3ef6254fb47a2f37e06bf not found: ID does not exist" Jul 16 05:41:01.001063 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:01.001045 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856"] Jul 16 05:41:01.004758 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:01.004737 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-e36375-openshift-default-58cb665687-7z856"] Jul 16 05:41:02.067350 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:02.067299 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dddb677c8-5d67z_408adabb-1c69-46f7-8c1b-de7b15045259/manager/0.log" Jul 16 05:41:02.075029 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:02.075008 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dddb677c8-5d67z_408adabb-1c69-46f7-8c1b-de7b15045259/copy-manifests/0.log" Jul 16 05:41:02.105907 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:02.105868 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3f65dab-2090-480d-b77b-64e7977d851e" path="/var/lib/kubelet/pods/d3f65dab-2090-480d-b77b-64e7977d851e/volumes" Jul 16 05:41:07.173794 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:07.173764 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678z892k_d36cdcda-f986-4d57-a141-12b5c455e8db/istio-proxy/0.log" Jul 16 05:41:07.299171 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:07.299128 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-prk6n_3ab1b99e-8fcc-400e-9537-357fa0e6a5c1/discovery/0.log" Jul 16 05:41:07.674308 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:07.674260 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-b6v4x_4fda6774-a044-441f-8357-afec2ddf322e/istio-proxy/1.log" Jul 16 05:41:08.617802 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:08.617772 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-z7n8w_9450e32b-d341-499e-8142-49addd0af661/main/0.log" Jul 16 05:41:08.987344 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:08.987261 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-cjdx9_df804050-3468-47a9-be52-161b28b87345/main/0.log" Jul 16 05:41:18.672209 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:18.672160 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-p97p6_67ee4d7c-8ae3-4f53-9a97-84a80cf5d67c/global-pull-secret-syncer/0.log" Jul 16 05:41:18.810099 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:18.810049 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9lpb5_1c5e6cb3-b901-4b3c-a82e-91aa7ccb0ecd/konnectivity-agent/0.log" Jul 16 05:41:18.829561 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:18.829500 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-187.ec2.internal_d36fd6545199dd402a0bbf93c6fc2c71/haproxy/0.log" Jul 16 05:41:26.936855 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:26.936831 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0f75d148-5b6a-4d27-949c-14e30653aaca/alertmanager/0.log" Jul 16 05:41:26.963518 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:26.963477 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0f75d148-5b6a-4d27-949c-14e30653aaca/config-reloader/0.log" Jul 16 05:41:26.985977 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:26.985952 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0f75d148-5b6a-4d27-949c-14e30653aaca/kube-rbac-proxy-web/0.log" Jul 16 05:41:27.008434 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:27.008407 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0f75d148-5b6a-4d27-949c-14e30653aaca/kube-rbac-proxy/0.log" Jul 16 05:41:27.033878 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:27.033830 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0f75d148-5b6a-4d27-949c-14e30653aaca/kube-rbac-proxy-metric/0.log" Jul 16 05:41:27.055261 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:27.055231 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0f75d148-5b6a-4d27-949c-14e30653aaca/prom-label-proxy/0.log" Jul 16 05:41:27.082259 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:27.082207 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0f75d148-5b6a-4d27-949c-14e30653aaca/init-config-reloader/0.log" Jul 16 05:41:27.456543 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:27.456520 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vk7m2_c2a0834f-b87b-454a-9ce3-75d41b90aead/node-exporter/0.log" Jul 16 05:41:27.477085 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:27.477064 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vk7m2_c2a0834f-b87b-454a-9ce3-75d41b90aead/kube-rbac-proxy/0.log" Jul 16 05:41:27.505801 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:27.505774 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vk7m2_c2a0834f-b87b-454a-9ce3-75d41b90aead/init-textfile/0.log" Jul 16 05:41:27.535351 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:27.535324 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-4rxlm_877ace0e-1bc5-468e-956d-96b8351dc1cf/kube-rbac-proxy-main/0.log" Jul 16 05:41:27.556544 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:27.556522 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-4rxlm_877ace0e-1bc5-468e-956d-96b8351dc1cf/kube-rbac-proxy-self/0.log" Jul 16 05:41:27.579317 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:27.579282 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-4rxlm_877ace0e-1bc5-468e-956d-96b8351dc1cf/openshift-state-metrics/0.log" Jul 16 05:41:29.319664 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.319630 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4w7ht/perf-node-gather-daemonset-lplwz"] Jul 16 05:41:29.320090 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.319984 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d328857d-cee4-42a1-bd00-638fb7139a0b" containerName="istio-proxy" Jul 16 05:41:29.320090 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.319998 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d328857d-cee4-42a1-bd00-638fb7139a0b" containerName="istio-proxy" Jul 16 05:41:29.320090 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.320006 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3f65dab-2090-480d-b77b-64e7977d851e" containerName="istio-proxy" Jul 16 05:41:29.320090 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.320012 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3f65dab-2090-480d-b77b-64e7977d851e" containerName="istio-proxy" Jul 16 05:41:29.320090 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.320023 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d328857d-cee4-42a1-bd00-638fb7139a0b" containerName="istio-proxy" Jul 16 05:41:29.320090 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.320028 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d328857d-cee4-42a1-bd00-638fb7139a0b" containerName="istio-proxy" Jul 16 05:41:29.320090 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.320033 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3f65dab-2090-480d-b77b-64e7977d851e" containerName="istio-proxy" Jul 16 05:41:29.320090 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.320037 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3f65dab-2090-480d-b77b-64e7977d851e" containerName="istio-proxy" Jul 16 05:41:29.320383 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.320105 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d328857d-cee4-42a1-bd00-638fb7139a0b" containerName="istio-proxy" Jul 16 05:41:29.320383 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.320112 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3f65dab-2090-480d-b77b-64e7977d851e" containerName="istio-proxy" Jul 16 05:41:29.323091 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.323075 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-lplwz" Jul 16 05:41:29.326008 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.325981 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4w7ht\"/\"openshift-service-ca.crt\"" Jul 16 05:41:29.326855 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.326841 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4w7ht\"/\"default-dockercfg-wmljj\"" Jul 16 05:41:29.326918 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.326880 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4w7ht\"/\"kube-root-ca.crt\"" Jul 16 05:41:29.329785 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.329766 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4w7ht/perf-node-gather-daemonset-lplwz"] Jul 16 05:41:29.446880 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.446847 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/410c48f7-e7bd-4fa7-97b3-6cae9e476bad-podres\") pod \"perf-node-gather-daemonset-lplwz\" (UID: \"410c48f7-e7bd-4fa7-97b3-6cae9e476bad\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-lplwz" Jul 16 05:41:29.447080 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.446913 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nn8t\" (UniqueName: \"kubernetes.io/projected/410c48f7-e7bd-4fa7-97b3-6cae9e476bad-kube-api-access-7nn8t\") pod \"perf-node-gather-daemonset-lplwz\" (UID: \"410c48f7-e7bd-4fa7-97b3-6cae9e476bad\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-lplwz" Jul 16 05:41:29.447080 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.446949 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/410c48f7-e7bd-4fa7-97b3-6cae9e476bad-sys\") pod \"perf-node-gather-daemonset-lplwz\" (UID: \"410c48f7-e7bd-4fa7-97b3-6cae9e476bad\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-lplwz" Jul 16 05:41:29.447080 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.446973 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/410c48f7-e7bd-4fa7-97b3-6cae9e476bad-lib-modules\") pod \"perf-node-gather-daemonset-lplwz\" (UID: \"410c48f7-e7bd-4fa7-97b3-6cae9e476bad\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-lplwz" Jul 16 05:41:29.447080 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.447052 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/410c48f7-e7bd-4fa7-97b3-6cae9e476bad-proc\") pod \"perf-node-gather-daemonset-lplwz\" (UID: \"410c48f7-e7bd-4fa7-97b3-6cae9e476bad\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-lplwz" Jul 16 05:41:29.548170 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.548124 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/410c48f7-e7bd-4fa7-97b3-6cae9e476bad-podres\") pod \"perf-node-gather-daemonset-lplwz\" (UID: \"410c48f7-e7bd-4fa7-97b3-6cae9e476bad\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-lplwz" Jul 16 05:41:29.548365 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.548193 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nn8t\" (UniqueName: \"kubernetes.io/projected/410c48f7-e7bd-4fa7-97b3-6cae9e476bad-kube-api-access-7nn8t\") pod \"perf-node-gather-daemonset-lplwz\" (UID: \"410c48f7-e7bd-4fa7-97b3-6cae9e476bad\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-lplwz" Jul 16 05:41:29.548365 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.548224 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/410c48f7-e7bd-4fa7-97b3-6cae9e476bad-sys\") pod \"perf-node-gather-daemonset-lplwz\" (UID: \"410c48f7-e7bd-4fa7-97b3-6cae9e476bad\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-lplwz" Jul 16 05:41:29.548365 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.548249 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/410c48f7-e7bd-4fa7-97b3-6cae9e476bad-lib-modules\") pod \"perf-node-gather-daemonset-lplwz\" (UID: \"410c48f7-e7bd-4fa7-97b3-6cae9e476bad\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-lplwz" Jul 16 05:41:29.548365 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.548321 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/410c48f7-e7bd-4fa7-97b3-6cae9e476bad-podres\") pod \"perf-node-gather-daemonset-lplwz\" (UID: \"410c48f7-e7bd-4fa7-97b3-6cae9e476bad\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-lplwz" Jul 16 05:41:29.548365 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.548320 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/410c48f7-e7bd-4fa7-97b3-6cae9e476bad-proc\") pod \"perf-node-gather-daemonset-lplwz\" (UID: \"410c48f7-e7bd-4fa7-97b3-6cae9e476bad\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-lplwz" Jul 16 05:41:29.548567 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.548380 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/410c48f7-e7bd-4fa7-97b3-6cae9e476bad-proc\") pod \"perf-node-gather-daemonset-lplwz\" (UID: \"410c48f7-e7bd-4fa7-97b3-6cae9e476bad\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-lplwz" Jul 16 05:41:29.548567 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.548406 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/410c48f7-e7bd-4fa7-97b3-6cae9e476bad-lib-modules\") pod \"perf-node-gather-daemonset-lplwz\" (UID: \"410c48f7-e7bd-4fa7-97b3-6cae9e476bad\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-lplwz" Jul 16 05:41:29.548567 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.548419 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/410c48f7-e7bd-4fa7-97b3-6cae9e476bad-sys\") pod \"perf-node-gather-daemonset-lplwz\" (UID: \"410c48f7-e7bd-4fa7-97b3-6cae9e476bad\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-lplwz" Jul 16 05:41:29.555891 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.555863 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nn8t\" (UniqueName: \"kubernetes.io/projected/410c48f7-e7bd-4fa7-97b3-6cae9e476bad-kube-api-access-7nn8t\") pod \"perf-node-gather-daemonset-lplwz\" (UID: \"410c48f7-e7bd-4fa7-97b3-6cae9e476bad\") " pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-lplwz" Jul 16 05:41:29.634416 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.634329 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-lplwz" Jul 16 05:41:29.762178 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:29.762151 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4w7ht/perf-node-gather-daemonset-lplwz"] Jul 16 05:41:29.764093 ip-10-0-135-187 kubenswrapper[2561]: W0716 05:41:29.764067 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod410c48f7_e7bd_4fa7_97b3_6cae9e476bad.slice/crio-8099ef632258242dd81d7aebf785e6cbf2aaa9e64136633cb6eda92db27f968b WatchSource:0}: Error finding container 8099ef632258242dd81d7aebf785e6cbf2aaa9e64136633cb6eda92db27f968b: Status 404 returned error can't find the container with id 8099ef632258242dd81d7aebf785e6cbf2aaa9e64136633cb6eda92db27f968b Jul 16 05:41:30.104173 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:30.104138 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-lplwz" event={"ID":"410c48f7-e7bd-4fa7-97b3-6cae9e476bad","Type":"ContainerStarted","Data":"4dec7a79c746532050c95fa52371f7eb886bcdecebde30fd4cef82c88b52a9ff"} Jul 16 05:41:30.104342 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:30.104189 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-lplwz" event={"ID":"410c48f7-e7bd-4fa7-97b3-6cae9e476bad","Type":"ContainerStarted","Data":"8099ef632258242dd81d7aebf785e6cbf2aaa9e64136633cb6eda92db27f968b"} Jul 16 05:41:30.104342 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:30.104226 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-lplwz" Jul 16 05:41:30.119514 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:30.119462 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-lplwz" podStartSLOduration=1.119445874 podStartE2EDuration="1.119445874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 05:41:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:41:30.11810646 +0000 UTC m=+3080.490229692" watchObservedRunningTime="2026-07-16 05:41:30.119445874 +0000 UTC m=+3080.491569079" Jul 16 05:41:32.353935 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:32.353909 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-q7ghj_abe48c82-b51a-47de-9586-6fb499424b6a/dns/0.log" Jul 16 05:41:32.377963 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:32.377925 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-q7ghj_abe48c82-b51a-47de-9586-6fb499424b6a/kube-rbac-proxy/0.log" Jul 16 05:41:32.502704 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:32.502669 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vkqpj_076bc066-399e-417e-a245-14833a939ca8/dns-node-resolver/0.log" Jul 16 05:41:33.095947 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:33.095918 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2lr5q_ffc2facd-a805-43cc-91d2-e35ffff459fd/node-ca/0.log" Jul 16 05:41:34.115370 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:34.115337 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678z892k_d36cdcda-f986-4d57-a141-12b5c455e8db/istio-proxy/0.log" Jul 16 05:41:34.237522 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:34.237486 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-prk6n_3ab1b99e-8fcc-400e-9537-357fa0e6a5c1/discovery/0.log" Jul 16 05:41:34.311911 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:34.311875 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-b6v4x_4fda6774-a044-441f-8357-afec2ddf322e/istio-proxy/0.log" Jul 16 05:41:34.341476 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:34.341441 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-b6v4x_4fda6774-a044-441f-8357-afec2ddf322e/istio-proxy/1.log" Jul 16 05:41:35.190221 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:35.190184 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n86pm_0e460403-6e6e-40d8-9a53-446614aba668/serve-healthcheck-canary/0.log" Jul 16 05:41:35.846585 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:35.846558 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bmns9_42adc88e-a3b3-42eb-a476-d796fb65e81e/kube-rbac-proxy/0.log" Jul 16 05:41:35.867373 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:35.867350 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bmns9_42adc88e-a3b3-42eb-a476-d796fb65e81e/exporter/0.log" Jul 16 05:41:35.888345 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:35.888323 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bmns9_42adc88e-a3b3-42eb-a476-d796fb65e81e/extractor/0.log" Jul 16 05:41:36.118809 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:36.118745 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4w7ht/perf-node-gather-daemonset-lplwz" Jul 16 05:41:39.083061 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:39.083027 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dddb677c8-5d67z_408adabb-1c69-46f7-8c1b-de7b15045259/manager/0.log" Jul 16 05:41:39.099860 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:39.099836 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dddb677c8-5d67z_408adabb-1c69-46f7-8c1b-de7b15045259/copy-manifests/0.log" Jul 16 05:41:40.527800 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:40.527762 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-8647fcdd7d-6kck6_ae569a1c-470c-41de-984c-ee5536109cae/manager/0.log" Jul 16 05:41:41.018244 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:41.018214 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678z892k_d36cdcda-f986-4d57-a141-12b5c455e8db/istio-proxy/0.log" Jul 16 05:41:41.128143 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:41.128109 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-prk6n_3ab1b99e-8fcc-400e-9537-357fa0e6a5c1/discovery/0.log" Jul 16 05:41:41.174786 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:41.174759 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-b6v4x_4fda6774-a044-441f-8357-afec2ddf322e/istio-proxy/0.log" Jul 16 05:41:41.207313 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:41.207281 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-b6v4x_4fda6774-a044-441f-8357-afec2ddf322e/istio-proxy/1.log" Jul 16 05:41:42.426812 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:42.426786 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678z892k_d36cdcda-f986-4d57-a141-12b5c455e8db/istio-proxy/0.log" Jul 16 05:41:42.542165 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:42.542135 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-prk6n_3ab1b99e-8fcc-400e-9537-357fa0e6a5c1/discovery/0.log" Jul 16 05:41:42.596542 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:42.596511 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-b6v4x_4fda6774-a044-441f-8357-afec2ddf322e/istio-proxy/0.log" Jul 16 05:41:42.625900 ip-10-0-135-187 kubenswrapper[2561]: I0716 05:41:42.625869 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-b6v4x_4fda6774-a044-441f-8357-afec2ddf322e/istio-proxy/1.log"